- Install LoadLeveler v2.2 or OpenPBS - Grant administrative access to account that will run Catalina. (see the LoadL or PBS admin guides) - Install Python v2.4.1+ - Download the most recent catalina..tar.gz file from http://www.sdsc.edu/catalina - Extract files: - gunzip -c catalina..tar.gz | tar xf - - cd catalina - There are 2 ways of installing Catalina: A. Interactive script B. Manual A. Interactive script ===================== 1 ./configure.sh It will prompt you for all necessary settings as in B, run conf.sh; make; make install; and optionally make config. It also lets you choose if you want to dump the settings to a file for later use. If the installation fail during any step, source the settings.sh created by this script, take corrective actions and retry. B. Manual =============== 1. set the following environment variables, for example : # CLUSTER_NAME {DS | TG} export CLUSTER_NAME=TG # PBS or LL export RESOURCEMANAGER=PBS # Resource manager library path export CATALINA_RMLIBDIRS=-L/usr/local/lib # Resource manager include path export CATALINA_RMINCDIRS=-I/usr/local/include # owner of Catalina files export CATALINA_CATOWNER=kenneth # group of Catalina files export CATALINA_CATLOCKGROUP=ops100 # directory in which to place man pages export CATALINA_MANPATH=/home/sdsc/kenneth/catalina/man # directory in which to install executables export CATALINA_INSTALLDIR=/home/sdsc/kenneth/catalina/install # directory in which to store database files export CATALINA_DBDIR=/home/sdsc/kenneth/catalina/db # directory in which to store archived logs export CATALINA_ARCHIVEDIR=/home/sdsc/kenneth/catalina/archive # C compiler export CC=/usr/bin/gcc # Python interpreter export CATALINA_PYTHONPATH=/paci/loadl/diag/python152/bin/python # permissions for user-settable reservation commands export CATALINA_CATUSERPERMS=2755 # default job class or queue export CATALINA_DEFAULT_JOB_CLASS=standard # If LL, location of LoadLeveler admin file export CATALINA_LOADL_ADMIN_FILE=/paci/loadl/LoadL_admin.AIX5L # If PBS, name of host from which jobs will be submitted and reservations set # If desired, this can be set to IGNORE. In that case, Catalina will not # consider the <@hostname> part of the jobowner field. All users will be # considered equivalent, regardless of host. #export CATALINA_PBS_SUBMITHOST=ctf01 export CATALINA_PBS_SUBMITHOST=IGNORE 2. Run conf.sh ./conf.sh Many other environment variables are available to customize the install. Check the conf.sh script for these. 3. Run make make 4. Run make install make install **************** - If this is the first install, generate a catalina.config file: - make config - cp catalina.config - cd - edit /catalina.config see comments in the file. - cd - ./initialize_dbs - start scheduling - start.ksh - show_q - show_res - show_bf - stop scheduling - stop.py Catalina Policies There are a number of policies available in catalina.config that allow the administrator some control of job scheduling. - MAXJOBPERUSERPOLICY/MAXJOBPERUSERCOUNT : When on, this limits the number of Running jobs per user. If this limit is reached, all Idle jobs go to nonqueued, and are not considered for scheduling. - MAXJOBQUEUEDPERUSERPOLICY/MAXJOBQUEUEDPERUSERCOUNT : When on, this limits the number of queued jobs per user. If this limit is reached, all other Idle jobs go to nonqueued, and are not considered for scheduling. - MAXJOBPERACCOUNTPOLICY/MAXJOBPERACCOUNTCOUNT : Limits Running jobs per allocations account. - MAXJOBQUEUEDPERACCOUNTPOLICY/MAXJOBQUEUEDPERACCOUNTCOUNT : Limits queued jobs per allocations account. - QOS_MAXJOBPERUSERPOLICY_STRING : For each QOS (quality of service), limits running jobs per user. - QOS_MAXJOBQUEUEDPERUSERPOLICY_STRING : For each QOS, limits queued jobs per user. - QOS_MAXJOBPERACCOUNTPOLICY_STRING : For each QOS, limits running jobs per account. - QOS_MAXJOBQUEUEDPERACCOUNTPOLICY_STRING : For each QOS, limits queued jobs per account. - QOS_MAXNODESECRUNNINGPERACCOUNTPOLICY_STRING : For each QOS, limits the total node-seconds of running jobs per account. - QOS_MAXNODESECQUEUEDPERACCOUNTPOLICY_STRING : For each QOS, limits the total node-seconds of queud jobs per account. - USER_SET_LIMITS_DICT_STRING : Sets limits on user-settable reservations. Allocation accounts can be limited by : number of reservation instances ('instances_int'), number of nodes in each instance ('nodes_int') and number of seconds in each instance ('seconds_int'). If an account is not explicitly listed, the limits in 'DEFAULT' will be applied. The 'GLOBAL' policies restrict resources used by all user_set reservations. 'window' is the sliding time window in seconds. For example, 86400 seconds means that no 24 hour window will have more than the limit's worth of node-seconds of user_set reservations. 'ABSOLUTE_LIMIT' is the maximum number of node-seconds permitted in any 'window'. 'REQUIREDFEATURESLIST' is a list of features that must be present on user_set reserved nodes. On heterogenous clusters, this would enable user_settable reservations on a subset of the nodes.