• Compute & Data Resources 
  • Gordon
  • Trestles
  • Triton
  • Software Applications
  • GPFS-WAN
  • HPSS User Guide
  • Visualization Services
• Consulting 
  • Submit a Ticket
  • Report a Security Incident
  • E-mail Us
  • User Responsibility
  • Using Your Account
  • Managing Your Account
  • Seminars & Publications
  • Using SSH at SDSC
  • Resource User Guides
• Training 
  • Upcoming Workshops
  • Open for Registration
  • Past Workshops
  • User Guides
  • Training References
  • Distance Learning Modules
• Allocations 
  • Allocations Home
  • Add/Remove Users
• Tools 
  • Add/Remove Users
  • Change Password
  • Forgot Password
  • Submit a Ticket
  • IPM Performance Monitoring

• XSEDE User News
• TAPP
• SAC
• Visualization Services
• About User Services

SDSC > User Support > Compute and Data Resource Guides > Gordon

• Gordon User Guide
• Technical Summary
• Access
• Modules
• Software Packages
• Accounts
• Compiling
• Running
• Storage
• vSMP
• Torque in Depth
• Monitoring
• Dedicated I/O Nodes
  • Tutorials
  • Hadoop
  • VisIt
  • MATLAB
    • Keypairs
    • Usage Examples

Gordon User Guide: Compiling

Gordon provides the Intel, Portland Group (PGI), and GNU compilers along with multiple MPI implementations (MVAPICH2, MPICH2, OpenMPI). Most applications will achieve the best performance on Gordon using the Intel compilers and MVAPICH2 and the majority of libraries installed on Gordon have been built using this combination. Although other compilers and MPI implementations are available, we suggest using these only for compatibility purposes.

All three compilers now support the Advanced Vector Extensions (AVX). Using AVX, up to eight floating point operations can be executed per cycle per core, potentially doubling the performance relative to non-AVX processors running at the same clock speed. Note that AVX support is not enabled by default and compiler flags must be set as described below.

Using the Intel compilers (Default/Suggested)

The Intel compilers and the MVAPICH2 MPI implementation will be loaded by default. If you have modified your environment, you can reload by executing the following commands at the Linux prompt or placing in your startup file (~/.cshrc or ~/.bashrc)

module purge
module load intel mvapich2_ib

For AVX support, compile with the -xHOST option. Note that -xHOST alone does not enable aggressive optimization, so compilation with -O3 is also suggested. The -fast flag invokes -xHOST, but should be avoided since it also turns on interprocedural optimization (-ipo), which may cause problems in some instances.

For more information on the Intel compilers: [ifort | icc | icpc] -help

Note for vSMP users on Gordon – MPI applications intended for use on the large memory vSMP nodes should use MPICH2 instead of MVAPICH2. This version of MPICH2 has been specially tuned for optimal vSMP message passing performance.

Note for C/C++ users: compiler warning - feupdateenv is not implemented and will always fail. For most users, this error can safely be ignored. By default, the Intel C/C++ compilers only link against Intel's optimized version of the C standard math library (libmf). The error stems from the fact that several of the newer C99 library functions related to floating point rounding and exception handling have not been implemented.

Using the PGI compilers

The PGI compilers can be loaded by executing the following commands at the Linux prompt or placing in your startup file (~/.cshrc or ~/.bashrc)

module purge
module load pgi mvapich2_ib

For AVX support, compile with -fast

For more information on the PGI compilers: man [pgf90 | pgcc | pgCC]

Using the GNU compilers

The GNU compilers can be loaded by executing the following commands at the Linux prompt or placing in your startup files (~/.cshrc or ~/.bashrc)

module purge
module load gnu openmpi

Notes and hints

• The mpif90, mpicc, and mpicxx commands are actually wrappers that call the appropriate serial compilers and load the correct MPI libraries. While the same names are used for the Intel, PGI and GNU compilers, keep in mind that these are completely independent scripts.
• If you use the PGI or GNU compilers or switch between compilers for different applications, make sure that you load the appropriate modules before running your executables.
• When building OpenMP applications and moving between different compilers, one of the most common errors is to use the wrong flag to enable handling of OpenMP directives. Note that Intel, PGI, and GNU compilers use the -openmp, -mp, and -fopenmp flags, respectively.
• Explicitly set the optimization level in your makefiles or compilation scripts. Most well written codes can safely use the highest optimization level (-O3), but many compilers set lower default levels (e.g. GNU compilers use the default -O0, which turns off all optimizations).
• Turn off debugging, profiling, and bounds checking when building executables intended for production runs as these can seriously impact performance. These options are all disabled by default. The flag used for bounds checking is compiler dependent, but the debugging (-g) and profiling (-pg) flags tend to be the same for all major compilers.