CSE-746 – Advanced Parallel and High Performance Computing
McMaster University
Winter 2024
Instructor: Dr. Sergey Mashchenko
Course outline: HTML
Important notes:
- Course project deadline is April 22 before midnight.
- Once you have Alliance account (see below), you need to use your Alliance credentials to login to
our Moodle site,
and then follow this link to enrol in the "2024W-Intro-OpenACC" course.
When prompted for a password, use CSE746 . This material will be used during our lectures 3 and 4.
- You will need a (free) Alliance (former Compute Canada) account for this course. This will be needed to access both some course
materials, and HPC resources for in-class work and home assignments. You will need to get an account before Lecture #3
(January 22, 2024).
- If you already have an account, just use it.
- If you don’t have an account, but your supervisor has one, ask them to sponsor you.
- If your supervisor doesn’t have an account, ask them to get one (it is free), and then sponsor you.
- As a last resort, I can sponsor some of you.
- How to get an Alliance account
- Once your supervisor has an account, ask them to provide their CCRI identifier (it looks like this: abc-123-01).
- Apply for your Alliance account on this page.
- Fill out the form, identifying yourself as a grad student. Provide your supervisor’s CCRI.
- Once submitted, your supervisor will get an email, which they have to act upon (click on a link to confirm you as their student). Ask them to check their SPAM/JUNK boxes.
- Get started with Alliance
- You are strongly recommended to attend one of our weekly New User webinars (every Tuesday at 2pm EST). Check our
Events calendar to register for one. (Alliance account is not
required, but you'll need a Zoom account.)
- Or watch an older recording on our youtube channel.
- Full wiki documentation is here.
- Instructions for home assignments
- Get an Alliance account
- Login to a cluster (graham, cedar) using your Alliance user name:
$ ssh user_name@graham.alliancecan.ca
- Allocate a GPU for interactive use (def-xxx is your supervisor's default account; run the command id
to figure it out):
$ salloc --time=0-00:30 -c 16 -A def-xxx --mem=64G --gres=gpu:p100:1
- Load the corresponding module (nvhpc for OpenACC, cuda for CUDA), e.g.:
$ module load nvhpc
- When done, don't forget to run exit to release the resource.
- If it takes too long to allocate a GPU with salloc, you can switch to another cluster, or try submitting a batch job
with sbatch job_script.sh command. The job script can look like this:
#!/bin/bash
#SBATCH --time=0-00:30
#SBATCH -A def-xxx # Replace with your supervisor's def account
#SBATCH -c 16
#SBATCH --mem=64G
#SBATCH --gres=gpu:p100:1
# Place your command(s) here:
./a.out
- For short (<10s) codes, improve timing measurement by running the code 10 times in a row non-stop, then taking the shortest
timing (can also be used inside a job script):
$ for ((i=0; i<10; i++)); do ./a.out | grep "time:"; done |sort
Lecture
notes (in PDF format):
- Jan. 8: Lecture 1 (Introduction to HPC computing: Overview, Parallel Computer Memory
Architectures, Parallel Programming Models)
- Jan. 15: Lecture 2 (Introduction to HPC computing: Designing Parallel Programs,
Parallel examples)
- Jan. 22: Quiz #1 (covers Lectures 1-2); Lecture 3: slides,
online materials, requires Alliance account,
use "CSE746" password when registering for the course (Introduction to GPUs, OpenACC)
- Jan. 29: Lecture 4: slides,
online materials, requires Alliance account,
use "CSE746" password when registering for the course (OpenACC contunued)
- Feb. 5: Lecture 5: Quiz 2 (covers Lectures 3-4), Assignment #1 given (OpenACC), slides (Introduction to CUDA)
- Feb. 12: Lecture 6: slides (CUDA basics)
- Feb. 19: no lecture (Family Day)
- Feb. 26: Lecture 7: Quiz 3 (covers Lectures 5-6), Assignment #2 given (OpenACC), slides
(CUDA code optimization; random numbers in CUDA)
- March 4: Lecture 8: slides (Matrix multiplication; streams)
- March 11: Lecture 9: Assignment #3 given (CUDA), slides (CUDA optimization
continued, CUDA on multiple GPUs)
- March 18: Lecture 10: slides (Thrust template library)
- March 25: Lecture 11: course project given (CUDA), slides (CuBLAS, new CUDA features,
profiling CUDA)
- Apr. 1: Lecture 12 (optional): GUI tools demo (debugging, profiling). Useful links:
Home
work
To
read PDF files on this page you will need the free Acrobat PDF Reader.