Material for CSC Computing Environment -course

1. Prerequisites (Accounts, Connecting, Basics of CLI)

1.1 Slides: Accounts and Projects

1.2 Video: Accounts and Projects

1.3 Slides: Connecting to CSC Computers

1.4 Video: Connecting to CSC Computers

1.5 Video tutorial: Connecting to Puhti

1.6 Tutorials and exercises

  1. Tutorial - Start with the CSC-account and project (essential)

  2. Tutorial - Login to Puhti with browser or SSH (essential)

  3. Tutorial - Basic linux commands

  4. Tutorial - Basic file editing

  5. Advanced tutorial - Use SSH-keys to authenticate connection

  6. Advanced tutorial - Run R studio/Jupyter Notebook on Puhti via ssh-tunnel and browser This requires ssh-keys (see above) but is the recommended way to use these interactive tools.

2. Introduction to HPC environment

2.1 Slides

2.2 Video: HPC Environment

2.3 Video: CSC Datacenter in Kajaani

3. Disk areas

3.1 Slides

3.2 Video: Disk Areas

3.3 Tutorials and exercises

  1. Tutorial - Main disk areas in CSC supercomputing environment (essential)

  2. Tutorial - Fast disk areas in CSC supercomputing environment

  3. Exercise - High I/O operation computing tasks (advanced)

4. Module system

4.1 Slides

4.2 Video: Modules and preinstalled software

4.5 Tutorials and exercises

  1. Tutorial - Modules in Puhti (essential)

  2. Advanced tutorial - Biosoftwares in Puhti

5. Batch queue system and interactive use

5.1 Slides

5.2 Video: Batch jobs

5.3 Tutorials and exercises

  1. Tutorial - Serial batch jobs (essential)

  2. Tutorial - Parallel batch jobs

  3. Tutorial - Interactive batch jobs

  4. Exercise - Retrieving data from bio data repositories (advanced)

  5. Exercise - Serial, array and parallel jobs with R + contours calculation from DEM with raster package (GIS)
  6. Exercise - Serial, array and parallel jobs with Python + NDVI calculation rasterio package (GIS)

6. Batch job resource usage

6.1 Slides

6.2 Video: Resources usage

6.3 Tutorials and exercises

  1. Tutorial - sacct and seff, resources used (essential)

  2. Exercise - Find your past job resource usage

7. Allas and where to keep your data

7.1 Slides

7.2 Video: Allas Object Storage

7.3 Video: Using Allas

7.4 Tutorials and exercises

  1. Tutorial - File transfer with Allas (essential)

  2. Tutorial - File backup with Allas

  3. Tutorial - Allas in batch jobs

  4. Advanced Tutorial - Using Allas (bio-data example)

8. Installing your own software

8.1 Slides

8.2 Video: Installing own software

8.3 Tutorials and exercises

  1. Tutorial - Installing binary applictions (essential)

  2. Tutorial - Installing a simple C code from source

  3. Tutorial - Installing R applictions and libraries

  4. Tutorial - Installing Python applications and libraries (essential)

  5. Tutorial - Installing Perl applictions and libraries

  6. Tutorial - Installing Java applictions

  7. Exercise - Installing own C, C++, or Fortran

9. Containers and Singularity

9.1 Slides

9.2 Video: Containers

9.3 Tutorials and exercises

  1. Tutorial - Singularity introduction start (essential)

  2. Tutorial - Singularity introduction continued

  3. Tutorial - Running applications installed as containers

  4. Exercise - Replicating a Conda environment

  5. Exercise - Creating singularity containers

  6. Advanced tutorial - How to get containers

10. How to speed up jobs

10.1 Slides

10.2 Tutorials and exercises

  1. Advanced tutorial - Gaussian with HyperQueue

Information

This project has received funding from the European High-Performance Computing Joint Undertaking (JU) under grant agreement No 951732.

All material (C) 2020-2021 by CSC -IT Center for Science Ltd.
This work is licensed under a Creative Commons Attribution-ShareAlike 3.0
Unported License, http://creativecommons.org/licenses/by-sa/4.0/