Dash User Guide: RAMFS
With the large amount of memory on the Dash vSMP node—768 GB (48 GB/node)—users may wish to write their output to memory to improve performance. This feature is only available on the vSMP node. Note: due to overhead caused by the operating system, 650 GB is the maximum memory available to users.
How to Access RAMFS
The memory storage based filesystem is mounted as /ramfs on each node. A user accessible directory named /ramfs/$PBS_JOBID is created when a job is submitted to the queuing system. This directory on the vSMP is a temporary storage resource and once a job completes, the directory is purged and space released to the system. If the data written to RAMFS is needed after the run has completed, users should incorporate instructions in their job script to copy the data from RAMFS to persistent storage (such as GPFS). Failure to do this will result in the loss of all data that was on the RAMFS.
Sample Script
#!/bin/bash #PBS -l nodes=1:ppn=8 #PBS -l walltime=HH:MM:SS #PBS -q large # ... Additional lines needed for efficient vSMP job execution skipped cd /ramfs/$PBS_JOBID # copy the execution environment to /ramfs cp $HOME/run/data . mpirun -np 8 ./mpi.ex # copy the simulation output back to persistent storage, e.g. GPFS cp -r /ramfs/$PBS_JOBID /gpfs-wan/scratch/$USER/out/
