Day 1 Session 1: Introduction, Gordon Overview
Day 1 Session 2: Gordon User Environment & Running Jobs
Day 1 Session 3: Compiling and Running Jobs on Gordon
Day 1 Session 4: Compiling and Running Jobs on Gordon (cont'd)
Welcome
Gordon/Dash Overview
> ssh username@dash-login.sdsc.edu
The user environment is managed using modules. The default environment on login and non-vSMP nodes includes Intel compilers and MVAPICH 1.0 optimized for non-vSMP nodes.
vSMP nodes have intel compilers and MPICH2 optimized for vSMP nodes.
>module li (displays available modules)
>which mpirun (displays path for default mpirun, /usr/apps/compilers/intel/mvapich-gen2multirail-1.0/bin/mpirun)
>module unload mvapich
>module li
>module load mpich2 (load module optimized for vSMP nodes)
Code for non-vSMP nodes will still run fine on VSMP nodes, but it won’t be optimized for them.
>which mpirun (displays path for mpirun optimized for vSMP nodes, /usr/apps/compilers/ScaleMP/mpich2/1.0.8/bin/mpirun)
>module avail (displays available modules)
Gordon uses the TORQUE Resource Manager (also known as PBS) with Moab Workload Manager.
Non-vSMP nodes accessible via “batch”, “express”, and “long” queues.
vSMP node accessible via “large” queue.
Download sample files from /work/workshop/ on Gordon. Create workshop directory in your home directory.
Verify you have the right modules loaded:
>module li
Currently Loaded Modulefiles:
1) intel/11.1 2) mvapich/1.2.7
Compile the MPI hello world code:
>mpicc -o hello_world hello_world.c
Verify executable has been created:
>ls -lt hello_world
-rwxr-xr-x 1 train61 gue998 465045 May 9 10:22 hello_world
Compile code (19:10 min)
>mpicc –o hello_world hello_world.c
#!/bin/csh
#PBS -q batch
#PBS -N job_batch
#PBS -l nodes=1:ppn=4
#PBS -l walltime=0:10:00
#PBS -o batch-run1.out
#PBS -e batch-run1.err
#PBS -V
#PBS -M your_email_address
#PBS -m abe
#PBS -A gue998
cd /work/workshop (change this to your directory)
mpirun -v -machinefile $PBS_NODEFILE -np 4 ./hello_world
Partitions
Compute side of Dash
vSMP
How to use flash memory and RAMFS
To compile the example code:
>ifort –o hello_vsmp -openmp hello_vsmp.f90
To submit the job:
>qsub hello_world_vsmp.sh
To check the job status:
>checkjob –v –v JOBID (displays job details, status, running/queued)
Example of job script jobscript-vsmp.sh:
#!/bin/sh
#PBS -q large
#PBS -N job_vsmp
#PBS -l nodes=1:ppn=4
#PBS -l walltime=0:10:00
#PBS -o vsmp-run1.out
#PBS -e vsmp-run1.err
#PBS -V
#PBS –M your_email_address
#PBS -m abe
#PBS -A sds122
cd /work/workshop
export LD_PRELOAD=/opt/ScaleMP/libvsmpclib/0.1/lib64/libvsmpclib.so
export PATH="/opt/ScaleMP/numabind/bin:$PATH"
export KMP_AFFINITY=compact,verbose,0,`numabind --offset 4`
export OMP_NUM_THREADS=4
./hello_vsmp
Mahidhar Tatineni resumes a hands-on session on Gordon, beginning with a Hello World example running on the VSMP node using OpenMP.
July 2, 2013
Software Stack Upgrade for Gordon
Comprehensive Application and OS Upgrade Scheduled for June 2013
Data Oasis and NFS Storage Not Affected
An operati...
more news...
10/26/10
-
10/28/10
"Grand Challenges in Data-Intensive Discovery" conference was held October 26 - 28, 2010 at the San Diego Supercomputer Center (SDSC) on the campus o...
more events...
