MSS Driver

From SRB

Note that some of the new SRB code that defines an MSS resource is required in the SRB client, non-MCAT server (on MSS host), and the MCAT-enabled Server. So all need to be updated to the version with MSS defined (SRB 3.4.1).

As noted in How_To_Add_A_Driver, SRB admins may need to add a ResourceType token for the new resource. For the mss driver, you need to add 'mss file system', or something similar (it must contain 'mss'). See the How_To_Add_A_Driver for additional information.

When creating an MSS resource, you must create the vault path to match your user's access to MSS. For example, using user 'schroedr' to run the SRB server, the MSS achive is /SCHROEDR; so you would define the resource path to be /SCHROEDR or /SCHROEDR/Vault1 or something like that. SgetR will, for example, show the phy_default_path to be /SCHROEDR/srbVault/?USER.?DOMAIN/?SPLITPATH/?PATH?DATANAME.?RANDOM.?TIMESEC

The MSS driver has a simple disk cache manager as part of it to manage the data cache that is needed to interface to MSS. This manager is called after an MSS file is closed. If the SRB Server and MSS driver are inactive, you can safely remove any or all files and directories in the cache. But don't modify any files in the cache, as the driver assumes that a file that exists in the cache is valid. The cache limits may occasionally be exceeded, but over time (if the driver is active) the cache usage should return to normal levels. Since MSS system requires all files to go through this cache, it is important to provide the driver with sufficient space. There are configuration options that can be set to adjust the cache management parameters:

If the file "mssConfig" exists in the "data" directory, it will be opened and read for parameters. maxSize is in kilobytes (KB). If the cache grows beyond the limits, the oldest files will (normally) be removed until the cache totals are under the limits again. However, if the oldest file (modify time) is too young (defined in mssFileDvr.c to be 60 seconds) it will not remove any file, since another process might be using it. If the debugLevel is 0 no debug messages are logged, 1-4(or more) logs increasing types of messages. The defaults can also be changed in the source file (see configMaxFiles, for example). Here's an example:

maxFiles 100
maxSize 100000000
cachePath /tmp/srbMssCache
readPw 123
writePw abc
classOfService usage=sdsc
debugLevel 0

The read and write passwords (readPw and writePw) are optional, but will be used on the mss command lines if present. If you set these, you should never change them because the same pair must be used to later remove a file and the read password is needed to read the file.

If classOfService is provided, it will be included (preceeded with -cl) on the msrcp command line. Currently, NCAR has asked us to use a usage=sdsc classOfService.

There are some optional configuration parameters that can be used to store with an alternative ClassOfService and retention period if the file is being stored into a logical path that begins with a specified string. For example:

specialCasePath /zonea/home/test.demo/backup 
specialCaseCOS usage=backup
specialCaseRetention 30

will store files that are going under the /zonea/home/test.demo/backup collection with a class of service of usage=backup and a retention period of 30 days.

For a few reasons, the interface to MSS is somewhat slow. First, all files have to be cached fully before being transferred into or out of MSS, so there is no overlap in SRB network I/O and MSS transfer I/O. Also, the mss commands themselves are fairly slow, taking a few seconds even for small files (at least on the test host, dataportal). In addition, since there is no direct in-memory C API (all the MSS API functions invoke MSS commands) there is more overhead in running those and some operations require multiple steps. For example, the mssFstat call is slow because it needs to close the file, write it to MSS and then do a stat (msls) to confirm the transfer. But we're hoping that for a deep-archive off-site backup, MSS will still work well.

The interface for creating directories is slow too, but this is avoided, at least for the most part. For other resources, when a create call is received, routines in extFunct.c will call the the driver's stat and mkdir functions to create the subdirectories. But for MSS resources, we have it skip this since it is unneeded and slow for MSS. But the MSS driver can create a directory, which is does by creating a file (.mkdir) to create MSS directories. So since MSS simulates directories this will work such that 'stat' calls will then succeed.

MSS does not allow zero-length files, but via special processing in the driver they are allowed for SRB MSS files. To do this, the driver stores a small file (2 bytes) instead and sets the comment field, via msscomment, to a special value. When doing a stat, the driver checks this field and returns 0 for the size if the size is 2 and this field is set. When moving a file from MSS to the cache, it also checks the size, and if it is 2, checks the comment and if the comment flag is set sets the file to 0 bytes. The rest of the handling proceeds as normal. If a zero-length file is appended to, no special processing is needed to reset the comment flag because replacing a file in MSS automatically clears the comment field.