Scommand Manpages c
From SRB
The following is a categorization of Scommands. This list was first produced by Raymond Turney and is reproduced here with his permission.
Purpose of Document
This document organizes references to the Scommand manual pages by the function of the commands.It also includes one line explanations of the purpose of the individual commands, so that novice users and administrators can easily find out which command they are likely to want.
The categories are:
- Initialization, Exit, Miscellaneous
- Virtual Unix File System Management
- Data Transfer
- Physical Storage Management
- Replica Management
- Metadata Manipulation
- Security Management
- Proxy Command
- Zone management
- Administration
There is a brief text description of each area, followed by a table or relevant commands listed in alphabetical order.The table lists each Scommand, by name with a link to the relevant man page, and a one line description of the purpose of the command to assist readers in quickly finding the command they want.
Some commands are listed more than once, because they are important in more than one context. Stoken, for example, is the primary tool for obtaining information about the current zone, and is also a general purpose tool for obtaining information about types of SRB metadata.Thus it is listed under both Metadata Manipulation and Zone Management.
It is suggested that users should generally learn the basics of Initialization, Virtual UNIX File System management, and Data Transfer first.All users will need to know how to start and stop an SRB session, get help, navigate the SRB’s virtual UNIX file system, and transfer data to and from the SRB.
What else a user will need to know will vary with the nature of his or her project.
Physical Storage Management is important for those users who are likely to need to control where their SRB stored data are physically located, or for users with a lot of small files.This section discusses the commands used in specifying where data will be stored.
Replica Management is useful for user who will have a number of different copies of their data stored in different places.The most obvious use for this is backup, but moving your data to a local storage system ahead of time can do much to improve performance.A grid user running on two separate machines, one in California and one in Texas, might want two copies of his or her data, one at each site.Replica management can help the user solve the associated problems.
The SRB has extensive support for user defined metadata, in addition to the system defined metadata.User whose applications have associated metadata would profit from investigating these.Digital Library users will be particularly interested in the Dublin Core support..
Security management is mostly for sites using the SRB ticketing system to authorize file sharing.Users who do not wish to share access to files with others using the ticketing system should probably focus on something else.
The command for running proxy commands will be interesting to users who want to run a program on the storage host to subset their data.Because shipping large datasets over the network can be very slow, users who want to work on identifiable subsets of a large data set, but who do not have appropriate user defined metadata, will be particularly interested in this.DataCuttersupport may also interest these users.
Zone management commands involve federating different SRB installations using zones.It is important for users of large grid environments involving multiple SRB installations.Unless the user is already working in a zoned environment, he or she may wish to postpone learning about these until after the other parts of the SRB have been mastered.
Administration deals with commands likely to be most useful to administrators, such as the commands for adding users and resources, etc.
Scommands Return to top of document
Contents |
Initialization, Exit, Miscellaneous
The first commands a user should learn are Sinit, which starts an SRB session, and Sexit, which stops an SRB session.Shelp, which gets a help list of commands and what they do, is also a useful thing to learn quickly.Serror gives a text explanation of a numeric SRB error.Srmtrash actually removes files which have been deleted, since by default the SRB uses the PC and Mac approach of moving deleted files to a trash bin, and then emptying the trash bin.Sconvert is a file format conversion program for image files.The other commands get information on SRB entities, such as containers, which do not correspond to anything in traditional hierarchical file systems.
|
Command |
Function |
|
Converts image data from one format to another | |
|
Displays SRB environment information | |
|
Provides text explanation of SRB numeric error message | |
|
Exits an SRB session | |
|
Gets information on SRB data objects | |
|
Gets information on SRB Resources | |
|
Gets a list of commands and brief explanations | |
|
Starts an SRB session | |
|
Empties trash bin |
Scommands Return to top of document
Virtual UNIX File System Management
SRB presents a virtual UNIX file system to the user, and offers functions to manage the virtual file system corresponding many of the traditional UNIX commands.Most of the commands below have direct counterparts in UNIX, and the reader is expected to know them, so they will not be discussed individually.
SgetColl gets information on SRB collections.The SRB concept of a collection corresponds most closely to the concept of a directory in a hierarchical file system, so SgetColl can be thought of as getting information about directories.
Containers considerably improve storage system performance when managing large numbers of small files on mass storage systems.The user first makes a container and then puts SRB data objects into it.Smkcont makes a container, Slscont lists containers.
Ssh is worth noting.It is a shell for the SRB virtual file system.When used independently on their own, each of the Scommands is a separate entity, and each sets up its own connection to the SRB.Ssh makes this unnecessary, and also supports local sh, ksh, and bash commands on the local host.So Ssh can considerably improve the performance of production scripts.
|
Scommand |
Function |
|
UNIX cat command for SRB virtual file system | |
|
UNIX cd command for SRB virtual file system | |
|
UNIX chmod command for SRB virtual file system | |
|
UNIX cp command for SRB virtual file system | |
|
UNIX ln command for SRB virtual file system | |
|
UNIX ls command for SRB virtual file system | |
|
Lists containers, or lists contents of a container | |
|
Makes a new container | |
|
UNIX mkdir command for SRB virtual file system | |
|
UNIX mv command for SRB virtual file system | |
|
UNIX passwd command for SRB virtual file system | |
|
UNIX pwd command for SRB virtual file system | |
|
UNIX rm command for SRB virtual file system | |
|
UNIX rmdir command for SRB virtual file system | |
|
Bourne and related shell compatible SRB shell |
Scommands Return to top of document
Data Transfer
These commands deal with getting data into and out of the Storage Resource Broker.The basic commands are Sget, which gets data from the SRB virtual file system, and Sput, which puts data into the SRB virtual file system.Users may wish to investigate the parallel data transfer options for these commands, (-m and –M) in the interest of faster data transfer.Sbload is the bulk loading command; a user with a lot of files to put into the SRB at the same time should use it (it reduces the overhead of loading each file substantially).Sbunload is the best command for getting a lot of files out of the SRB at once.
Registration is the process of telling the Storage Resource Broker that a file or directory exists, and providing the necessary metadata for the MCAT to allow access to the data via the SRB.Sbregister for bulk registration is a useful command to consider when setting up a new SRB installation and bringing a lot of data into the SRB in a hurry.Most users will not need to worry about registration, however.
The other commands listed under this heading are less often used, and most users can safely defer learning about them.
|
Scommand |
Function |
|
Appends local file or SRB object to SRB object | |
|
Bulk loads files into the SRB system | |
|
Gets a lot of files out of the SRB at once | |
|
Registers a lot of files with the SRB, quickly | |
|
Runs a checksum on data in the SRB | |
|
Equivalent of FTP get command for SRB space | |
|
Equivalent of FTP put command for SRB space | |
|
Registers files and directories in the MCAT, so they appear in SRB virtual file system | |
|
Synchronizes replicas |
Scommands Return to top of document
Physical Storage Management
It is usually a good thing that the SRB virtualizes physical storage, so that the user does not have to worry about machine names, UNIX file systems, etc.Occasionally, it can be important to be able to control how and where the data is physically stored. These commands make it possible to manage physical storage of data via the SRB.This is most likely to be useful to two groups of users.
The first group is users with access to widely distributed data, or data geographically distant from the site of their computation.These users may wish to control where the SRB physically puts their data.Schdefres and Sphymove are most likely to be of interest to these users.
The second group is users with a lot of small files, and access to a hierarchical storage management system, such as HPSS, SAM-QFS, or DMF.The container feature of the SRB allows SRB users to associate a lot of small files so that the storage management system gets all of them from tape, whenever any of them are requested.This avoids the performance cost of separately requesting that each of several hundred small files be retrieved, reducing it instead to the cost of one retrieval request.
|
Scommand |
Function |
|
Changes default resource | |
|
Changes default host | |
|
Lists containers, or lists contents of a container | |
|
Makes a new container | |
|
Moves data objects or collections from one resource to another | |
|
Replicates a container | |
|
Removes a container | |
|
Stages file from tape to disk (Sun SAM-QFS only) | |
|
Synchronizes replicated containers |
document Return to top of document
Replica Management
Replica management is important in backing up large data sets and in grids where multiple copies of the data may be being used, and it is important to keep the data in a consistent state.
Sreplicate is the basic replication command, with Sreplcont for replicating containers.Synchronization is the act of putting multiple replicas in the same state.The basic command for synchronization is Srsync, with Ssyncd used for synchronizing individual SRB objects.
|
Scommand |
Function |
|
Replicates a container | |
|
Replicates an SRB object or collection | |
|
Synchronizes two copies, one of which may be in the local file system | |
|
Synchronizes multiple SRB replicas | |
|
Synchronizes containers |
Scommands Return to top of document
Metadata Manipulation
This includes both system metadata, such as who owns which SRB object, which physical storage resource it is located on, etc, and user defined metadata.SRB supports multiple types of user defined metadata.Annotations are notes attached to the data object; they can be anything but are not searchable.There are also fields reserved for user defined metadata in the MCAT SQL database, these have to obey the SQL rules for the fields provided, but can be queried using the Squery command.
Most users will not need to directly modify system metadata.In fact, they are probably best off not directly modifying system metadata, but instead relying on the appropriate Scommand.
Key commands for users considering use of the MCAT SQL database to store information about their data are Sufmeta and Squery.Sufmeta is used for modifying user defined metadata, Squery for querying metadata.The fields available to a user, and their SQL definitions, are discussed in the man page for Sufmeta.
|
Scommand |
Function |
|
Adds notes to SRB object | |
|
Lists the MCAT attributes | |
|
SRB backup utility | |
|
Modifies User defined metadata | |
|
Modifies system metadata for a collection | |
|
Modifies system metadata for SRB objects | |
|
Modifies User defined metadata in extensible rows | |
|
Queries SRB metadata | |
|
Displays information about SRB native types | |
|
Inserts, deletes and modifies attribute – value- unit triplets of user defined metadata |
Scommands Return to top of document
Security Management
Most of the security management commands are connected with the SRB ticket functionality.This functionality allows a user to share his or her data with another user on a temporary basis; by issuing the other user a “ticket”.Users who are not planning to use this functionality can postpone learning about it until later.
The exceptions are Spasswd, also discussed above as a Virtual UNIX command, and Sauth.
|
Scommand |
Function |
|
Creates an SRB password file | |
|
Displays SRB environment information, including security model used | |
|
Displays information on SRB tickets | |
|
UNIX passwd command for SRB virtual file system | |
|
Removes SRB ticket | |
|
Displays files read from SRB space for a ticket user | |
|
Issues tickets for SRB objects and collections | |
|
Displays objects and sub collections for a given ticket |
Scommands Return to top of document
Proxy Command
There is a command to access an SRB server and run scripts or programs on the server host.This may be valuable to users considering using the Data Cutter software to subset their data prior to transmission over the network
Users who are also familiar with Globus security and GSI should note that the SRB proxy command is unrelated to Globus proxies.The need for a proxy command comes from the fact that SRB stores data in storage resources as user SRB, so that users do not need accounts on the storage machines.As a result, the SRB must run scripts and programs on the storage systems for the user as user SRB.When doing this, SRB acts as a proxy for the user.
|
Scommand |
Function |
|
Runs a script or program for the user on the SRB server host |
Scommands Return to top of document
Zone management
These commands help in managing SRB zones.They can be ignored by users who do not have access to multiple zones, and even users who do have access multiple zones should focus on basic SRB functionality first.
|
Scommand |
Function |
|
Pulls metadata across zones | |
|
Pushes metadata across zones | |
|
Queries zone information | |
|
Inserts and modifies information about SRB zones | |
|
Synchronizes from other zones |
Scommands Return to top of document
Administration
Many, perhaps most, SRB commands are used by administrators.Some stand out as unusually likely to be important for the beginning SRB administrator.One of the first things that will need to be done will be to add hosts, resources, users and groups: the relevant commands for this are SingestToken, SingestUser and SingestGroup.New administrators should also consider using the SRB Java admin tool.
|
Scommand |
Function |
|
SRB backup utility | |
|
Deletes SRB entry | |
|
Displays SRB environment information | |
|
Provides text explanation of SRB numeric error message | |
|
Exits an SRB session | |
|
Gets information on collections | |
|
Gets information on SRB data objects | |
|
Gets information on SRB resources | |
|
Gets information on SRB tickers | |
|
Gets information on SRB users | |
|
Gets a list of commands and brief explanations | |
|
Adds a new group to the MCAT, and thus the SRB system | |
|
Adds a new token (SRB native metadata type) to the MCAT | |
|
Adds anew user to the MCAT, and thus the SRB system | |
|
Modifies data for a user in the MCAT, and thus the SRB system | |
|
Modifies data for a resource in the MCAT, and thus the SRB system | |
|
Pulls metadata across zones | |
|
Pushes metadata across zones | |
|
Registers files created outside SRB but accessible to SRB | |
|
Registers new location in SRB | |
|
Registers new resource in SRB | |
|
Empties trash bin | |
|
Synchronizes from other zones |
