2011 "Get Ready For Gordon/vSMP" Summer Institute

Day 1 Contents

Day 1 Session 1: Introduction, Gordon Overview

Day 1 Session 2: Gordon User Environment & Running Jobs

Day 1 Session 3: Compiling and Running Jobs on Gordon

Day 1 Session 4: Compiling and Running Jobs on Gordon (cont'd)




Day 1 Session 1: Introduction, Gordon Overview

View recording

View slides

(00:00) Bob Sinkovits, Introduction

(00:30) Mike Norman, SDSC Director, Gordon co-PI

Welcome

(03:30) Bob Sinkovits, SDSC Gordon Applications Lead

Gordon/Dash Overview


Day 1 Session 2: Gordon User Environment & Running Jobs

View recording

View slides

Mahidhar Tatineni conducts a live SSH session.

(05:00) Logging in to Dash:

Logging in to Dash

> ssh username@dash-login.sdsc.edu

The user environment is managed using modules. The default environment on login and non-vSMP nodes includes Intel compilers and MVAPICH 1.0 optimized for non-vSMP nodes.
vSMP nodes have intel compilers and MPICH2 optimized for vSMP nodes.

>module li (displays available modules)

>which mpirun (displays path for default mpirun, /usr/apps/compilers/intel/mvapich-gen2multirail-1.0/bin/mpirun)

>module unload mvapich

>module li

>module load mpich2 (load module optimized for vSMP nodes)
Code for non-vSMP nodes will still run fine on VSMP nodes, but it won’t be optimized for them.

>which mpirun (displays path for mpirun optimized for vSMP nodes, /usr/apps/compilers/ScaleMP/mpich2/1.0.8/bin/mpirun)

>module avail (displays available modules)

(14:20) Gordon: Running Jobs

Running Jobs

Gordon uses the TORQUE Resource Manager (also known as PBS) with Moab Workload Manager.

Non-vSMP nodes accessible via “batch”, “express”, and “long” queues.

vSMP node accessible via “large” queue.

Download sample files from /work/workshop/ on Gordon. Create workshop directory in your home directory.

Verify you have the right modules loaded:

>module li
Currently Loaded Modulefiles:
1) intel/11.1      2) mvapich/1.2.7


Compile the MPI hello world code:

>mpicc -o hello_world hello_world.c

Verify executable has been created:
>ls -lt hello_world
-rwxr-xr-x 1 train61 gue998 465045 May  9 10:22 hello_world


Compile code (19:10 min)
>mpicc –o hello_world hello_world.c


(19:40) Create Job Script

Job Script
#!/bin/csh
#PBS -q batch
#PBS -N job_batch
#PBS -l nodes=1:ppn=4
#PBS -l walltime=0:10:00
#PBS -o batch-run1.out
#PBS -e batch-run1.err
#PBS -V
#PBS -M your_email_address
#PBS -m abe
#PBS -A gue998
cd /work/workshop                           (change this to your directory)
mpirun -v -machinefile $PBS_NODEFILE -np 4 ./hello_world

Partitions

Compute side of Dash

vSMP

How to use flash memory and RAMFS

(29:20) Compiling and Running a Fortran OpenMP Example

To compile the example code:

>ifort –o hello_vsmp -openmp hello_vsmp.f90

To submit the job:

>qsub hello_world_vsmp.sh

To check the job status:

>checkjob –v –v JOBID (displays job details, status, running/queued)


Day 1 Session 3

View recording

View slides

Mahidhar Tatineni resumes a hands-on session on Gordon, beginning with a Hello World example running on the VSMP node using OpenMP.

(00:00) Compiling and Running a Fortran OpenMP Example (cont’d)

Compiling and running jobs


Example of job script jobscript-vsmp.sh:

#!/bin/sh

#PBS -q large

#PBS -N job_vsmp

#PBS -l nodes=1:ppn=4

#PBS -l walltime=0:10:00

#PBS -o vsmp-run1.out

#PBS -e vsmp-run1.err

#PBS -V

#PBS –M your_email_address

#PBS -m abe

#PBS -A sds122

cd /work/workshop

export LD_PRELOAD=/opt/ScaleMP/libvsmpclib/0.1/lib64/libvsmpclib.so

export PATH="/opt/ScaleMP/numabind/bin:$PATH"

export KMP_AFFINITY=compact,verbose,0,`numabind --offset 4`

export OMP_NUM_THREADS=4

./hello_vsmp

(01:00) View output and error files


Day 1 Session 4

View recording

View slides


Mahidhar Tatineni resumes a hands-on session on Gordon, beginning with a Hello World example running on the VSMP node using OpenMP.

(00:00) Compiling and Running a Fortran OpenMP Example (cont’d)

Recent Gordon News & Events

Gordon Ranks No. 88

March 30, 2014
Still One of World's Fastest HPC Systems Gordon, the unique data-intensive flash-based HPC resource at the San Diego Supercomputer Center (SDSC) at t... more news...


Past Events


Quick Links