Skip to content

Getting Started with HSI (HPSS Interface)

HSI is a FTP-like interface to the High Performance Storage System (HPSS). HSI supports wildcards for local and HPSS pathname pattern matching, and provides recursion for many commands, including the ability to store, retrieve, and list entire directory trees, or change permissions on entire trees. It may be used interactively or in batch mode, and may be included in UNIX pipes.

HSI is especially useful for SDSC users with accounts on multiple platforms, as it provides an interface to the SDSC HPSS system from a variety of SDSC machines. For more information on HSI commands, type help while running HSI interactively.

Back to Top

Authentication

DCE Keytabs

DCE Keytabs was used as the main authentication method for hsi in the past. Currently, Kerberos is the default method, implemented along with an HPSS software upgrade in April 2008.

Kerberos Principals

Please visit https://passive.sdsc.edu/, login with any of your SDSC machine passwords, follow the instructions to create your new HPSS Kerberos principal and set a password for the principal. Then using this password you can access HPSS in one of the following ways:

  1. More secure way and default way starting April 30th: Use this password every time you access HPSS, and hsi & htar programs will prompt you to enter your password.
  2. Password-less (ideal for scripts & automation): Use this password to obtain your Kerberos keytab (by running get_hpss_kb_keytab script on SDSC machines), and then onwards hsi & htar programs will use the keytab and not ask for the password.

Back to Top

Remote Site Execution

The old executables that were previously available for download in this section do not work with Kerberos. New versions will be available shortly.

Please contact consult@sdsc.edu for additional help.

Back to Top

Interactive Execution

To begin a session:

In order to use HSI interactively, users must log on to one of the SDSC hosts (for example, DataStar or an SDSC workstation) and type hsi at the command prompt. Depending on the .hsirc file, a Kerberos principal and password may be required.

Users may override the authentication method specified in their .hsirc file and select another authentication method by issuing one of the following commands to start an interactive HSI session:

hsi -A kerberos (for Kerberos authentication)
OR
hsi -A keytab (for DCE keytab authentication)

After successful authentication, the HSI utility responds with a confirming message and a "?" default HSI prompt (to customize the HSI prompt, see the HSI Reference Manual). For a list of HSI Commands, see the SDSC Storage Group's HSI Commands by Function. Remember to wait for the ? prompt before entering commands.


Back to Top

To end a session:

To end an interactive session, enter one of the following commands from the HSI prompt:

quit
exit
end
bye

Back to Top

Single-Line Execution

HSI also allows users to enter commands on the initial execute line. After pressing <RETURN>, each of the commands is executed, and HSI automatically ends. The bye or quit commands are not required to terminate HSI in single-line execution. Command line for single-line execution is:

hsi command1 [;command2;command3;...]

It may be necessary to use single or double quotes around metacharacters to avoid having the shell expand the metacharacters (such as the semicolon, which the shell uses to separate its own commands). Normal conventions for metacharacters apply. For example:

hsi get *.c

will not work, but

hsi get "*.c"

will retrieve all files ending in .c.

Note that for get and put operations, HSI uses a different syntax than FTP to identify the local file name. The syntax uses a ':' (colon character) to separate the local pathname from the HPSS pathname. For example:

hsi put local_file : hpss_file

Back to Top

Moving Large Files

To retrieve files larger than 100 GB, users may wish to use the -S option for the get or mget commands. The -S option allows the user to bypass staging large files on the disk cache. The syntax is:

get -S [other options] local : hpss [local : hpss ...]

For more information on the -S flag and other options, type get.

Back to Top

Archiving Files With HTAR

Htar is a utility for storing large numbers of files and or directories. It can be viewed as an HPSS-aware version of the familiar UNIX utility "tar". One way to move a large number of files and/or directories into HPSS is a multi-step process:

  1. Create a tar file:
    tar -cvf mytar.tar file_or_directory
  2. Store tar file in HPSS:
    hsi put mytar.tar

Htar combines this process into one command:

htar -c -f mytar.tar -L file_or_directory

Htar has been optimized for creation of archive files directly in HPSS. In most cases, it will be significantly faster to use htar to create a tar file in HPSS than to create it on local disk and then copy it to HPSS, or to use tar piped into ftp (or hsi) to create the tar file directly in HPSS. The program uses multiple threads and a sophisticated buffering scheme in order to package member files into in-memory buffers, while making use of the high-speed network striping capabilities of HPSS.

The complete documentation for htar is at: http://www.sdsc.edu/us/resources/hpss/htar.html.

Back to Top

Need Help?

More information on HSI and HSI commands is available at http://www.sdsc.edu/us/resources/hpss/hsi.html or contact SDSC Consulting.


Did You Get
What You
Wanted?
Yes No
Comments