9. Kara02 documentation

A collection of Topics specific to working with the kara02 HPC Cluster.

Note

If you are looking for Documentation for the original Kara Cluster please see: https://hpcwiki.eps.surrey.ac.uk/doku.php?id=atihpc:kara

9.1. Kara02 cluster overview

Kara02 is the HPC Cluster based within the Advanced Technology Institute (ATI).

The Kara02 cluster works on the Linux/Centos 7 operating system. The compute resources currently on kara02 can be utilised for a wide variety of workloads, from large parallel jobs, high memory and some GPU capabilities too. It is made up of 3 types of nodes listed below and has an intel truescale infinband network fabric:

../_images/kara-specs-list.png

Cluster Hardware Specifications (May not be up to date)

../_images/kara-diagram.png

Cluster topology diagram. Colour coded to match the hardware Specifications list above.

Software

Kara02 hosts a wide variety of software and development tools relevant to Science, Engineering and Statistical computing workloads. Compilers such GCC, scripting languages R and Python as well as a wide range of software such as meep, MPB, mathematica and many more…

If there is an application you need installed on the cluster please submit a request. See Request new software

If you have your own builds of software, you are free to build and use your own version in your hpc storage areas.

9.2. Kara02 quick start

To help users get started quickly, we have created a repository of working submission script examples for a variety of programs currently on the cluster, and hosted them on GitLab.

If you would like to contribute any examples scripts yourself please let us know.

The repository can be accessed at: https://gitlab.surrey.ac.uk/rcs/kara02-examples

If you are logged into kara and have setup your GitLab account and configured SSH access to it, you can clone the repository into your space, with the command below:

$ git clone git@gitlab.surrey.ac.uk:rcs/kara02-examples.git

The repository contains a variety of scripts:

  1. Example starter scripts (ready to be customised):

    • Interactive jobs

    • Raw Submission scripts

    • Raw Array job scripts

  2. A few ready made submission scripts & example inputs for specific software:

    • Meep & MPB

    • matlab

    • mathematica

    • and more…

  3. Introductory Example “hello world” exercise to get started with submitting jobs to kara.

9.3. Slurm (job scheduling)

Kara02 uses the Slurm job scheduler. See Slurm - job scheduler for more info.

On kara02 there are 2 different partitions (Queues) to where you can submit your jobs kara, and kara-gpu. The kara partition contains most nodes on kara02 these are the 32 CPU compute nodes plus the 6 High-Mem nodes with a large mem/core ratios, kara-gpu contains the 2 GPU compute nodes.

The configurations of the partitions are summarised in the table below:

Name

Node Count

Time limit

Purpose

kara-gpu

1 GPU Compute

1 day Default

Run jobs that require GPU cards.

kara

6 CPU Compute

1 day Default

Day to Day production jobs.

Your main decision when submitting jobs to the cluster is which partition to send your jobs to. Either kara or kara-gpu.

Depending on your jobs and what is requested your jobs will automatically be taken to the high-mem nodes, if you have asked for memory requirements beyond that of the standard CPU compute nodes.

To see the sets of features and whats available, the command showcluster can be used to give a summary of all nodes, the number of cores they each have, their memory and the partitions they belong.

[abc123@kara-login(kara) ~]$ showcluster
PARTITION  NODES  CPUS  MEMORY  NODES(A/I)
kara       6      28    128653  0/6
kara-gpu   1      16    64149   0/1

9.4. X2GO (Kara graphical desktop)

Note

Kara02 is now also available via the RemoteLabs web portal web portal

X2go provides a way get a desktop on the kara02’s login node, this can be useful if you need to do lightweight GUI work on the cluster, X2go is a remote desktop system base on X11 so it may be “slow and laggy” when used.

Caution

When using X2go you are on the login node, not on Kara02’s compute nodes, so do not run simulations on here.

9.4.1. X2GO setup

To use X2go you need to download the X2go client from the following website https://wiki.x2go.org/doku.php/download:start

It can be installed on Linux, Windows or Mac.

9.4.2. Connect to Kara02 with X2GO

Launch X2go, and setup a new session to kara02:

  • Setup up a new session, with the settings as below with your own username:

    ../_images/pref-x2go.png

  • You will then have a connection to select to kara02:

    ../_images/con_x2go.png

  • Once successfully connected you’ll given a pop put window with a desktop on kara02’s login node.

    ../_images/desktop_X2go.png