What can SRB do for you?
SRB provides an interface to data collections located on one or more storage systems. This interface allows you to interact with your data from multiple locations without needing to know the underlying structure or physical location of the storage systems. In addition, the use of metadata permits detailed descriptions of data, allowing you to search your collections. This page provides some usage scenarios of SRB in action.
Suggested Uses
Walk-throughs
- Managing collections
- User/group access control
- Replicating
- Parallel (faster) I/O
- System metadata
- User metadata
- Bulk Ops
- Registering
- Checksums
- Sync local and SRB collections
- Tickets
- Containers
Remote data management
- Don't know UNIX? SRB provides remote data management via Java, Windows, and web interfaces.
- Don't want to have to manage several copies of the same dataset? SRB can manage the "master" copy, ensuring that you have access from anywhere at any time, seeing that data as one file hierarchy even though data possibly resides on multiple physical resources. SRB has sync functionality to add new partial datasets.
- Don't want to have to worry about physical systems? As machines (hostnames, paths, OS's, etc.) and technologies come and go, your SRB interface and paths will not change.
- Need a high degree of fault tolerance? You can tell SRB to replicate your data to multiple physical resources, and you can store and compare checksums of your data.
Collaboration and public access
- Need to collaborate with several sites? SRB can easily enable collaboration by providing users at each site access to the same file hierarchy.
- Need to make your data public? Any web server anywhere can easily become a SRB client. You decide if access is to be restricted or anonymous.
- Need to make it easy to find the "needle in the haystack"? SRB has user definable metadata. You can tag files and folders with whatever information may be useful: Gender='Male', Author='John Doe', Color='blue', Age='35'. Any attribute value pair that may be helpful later for searching the data. Example: Show me all data where Gender = 'Male' and Age < '40'.
Managing collections
A collection is a SRB term for a virtual directory. This example shows how to create a new collection, put a file into the collection, then delete the file and the collection.
Start a new SRB session
% Sinit
Change to the directory you want to copy
% Scd
Print working directory
% Spwd
Make new directory (if necessary)
% Smkdir collection1
Change working directory to the new directory that was just created
% Scd collection1
Copy your file to the current directory
% Sput AFileName
Delete the new file (if necessary)
% Srm AFileName
Move up one directory level
% Scd ..
Remove the directory you created above
Srmdir collection1
User/group access control
View the man page for the Sls command, making note of the -C parameter
% Sman Sls
View the man page for the Schmod command for more details
% Sman Schmod
Use Java admin tool to look at user/group definitions and to create new user 'user2', new group 'group2', making user2 a member of group2.
% Sput AFileName
% Schmod r user2 demo AFileName
% Sls -C
If you want, you can connect as user2 and see that you can Sget the file
% Schmod n user2 demo AFileName
% Sls -C
Note it is 'groups' not the domain 'demo'
% Schmod n group2 groups AFileName
% Sls -C
Could verify that file can be accessed by group members
% Srm AFileName
Replicating
Put file into the current working directory
% Sput AFileName
This will replicate into the same resource, although one would typically replicate into another physical resource
% Sreplicate AFileName
% Sls
% Sls -l
Display information about the data object
% SgetD AFileName
% Sls -l -A "COPY = 1"
View man page on Sreplicate command
% Sman Sreplicate
Parallel (faster) I/O
Copy the Scommands to another computer (or build them there), and copy your ~/.srb/.MdasEnv and .MdasAuth files
From this second computer initialize your SRB session
% Sinit
View the man page for Sput, paying attention to the -v and -m parameters
% Sman Sput
Find large files to put
% Sput -v largefile
% Sput -v -m largefile largefile2
Remove your local files and purge files from the trash
% Srm largefile largefile2
% Srmtrash
Repeat with other files, notice transfer rates and number of threads
System metadata
Copy larger file to AnotherFileName
% Sput AnotherFileName
% Sls -l -A "SIZE > 2000"
% Sls -l -r -A "SIZE < 2000"
Display the MCAT attributes used to select SRB objects
% Sattrs
User metadata
Note: In some cases, the single quotes (') around the values are not needed, but normally they are.
% Smeta AnotherFileName
% Sufmeta name1 val1 AnotherFileName
% Smeta AnotherFileName
% Sls -A "ATTRCONDD name1 = 'val1'"
% Sls -A "ATTRCONDD name1 = 'val2'"
% Sls -A "ATTRCONDD aaa = 'val2'"
% SgetD -A "ATTRCONDD name1 = 'val1'" '*'
% Sget -A "ATTRCONDD name1 = 'val1'" '*'
% Sls -c "ATTRCONDD name1 = 'val2'"
Display the MCAT attributes used to select SRB objects
% Sattrs
% Sufmeta
Bulk Ops
% Sman Sbload
View man page for Sput command, making note of the -b parameter
% Sman Sput
(now, either use Sput . . .
% time Sput -b something* collection2
. . . or Sbload something* collection2)
% Sls
% Srm -r collection2 (notice it is fast)
% Srmtrash (notice it takes a while, doing actual rm)
time Sput something* collection2 (notice much slower without -b)
Registering
% Sman Sregister
% Sman Sbregister
% SgetD a-file (notice path)
(find a subdirectory to try)
% Sbregister full-path-of-directory Collection3
% Scd Collection3
% Sls -l
% Sget a-file
% Srm a-file (that you no longer want in both SRB and local file system)
% Srmtrash
% ls (see that it is gone)
% Srm -u another-file
% Srmtrash
% Sls (see it is gone in SRB)
% ls (see it is still there in local file system)
Checksums
View man page for Sput, making note of the -k and -K parameters
% Sman Sput
% Scd collection2
Perform checksum operation on your files
% Schksum -l -r
% Schksum -f -r
% Schksum -l -r
% Schksum -l filename
Sync local and SRB collections
Find a subtree to use, then move up one directory level
% Scd
% Sput -r -b -k subdirname subdirname
% Schksum -l -r subdirname
% Sman Srsync
Change or add a local file
% Srsync -l -r subdirname s:subdirname
% Srsync -l -r subdirname s:subdirname
Remove a local file
% Srsync -v -r subdirname s:subdirname
% Srsync -l -r subdirname s:subdirname
% Srsync -v -r subdirname s:subdirname
(notice how it does not remove the SRB copy)
Tickets
View man page for Sticket command
% Sman Sticket
User ticketuser@sdsc can use the ticket. By using the special ticketuser@sdsc, non-users can access the file via Stls and Stcat.
% Sticket -D filename ticketuser@sdsc
% Sman SgetT
% Sman Stls
% SgetT
% Stls ticket-value
% Sman Stcat
See that it is the same data
% Stcat -t ticket-value host > newfilename
At this point, a SRB or non-SRB user from any client host should be able to access the file via the Stcat command and the ticket-value
% Srmticket ticket-value
Containers
% Sman Slscont
% Sman Smkcont
% Sman Sreplcont
% Sman Srmcont
% Sman Ssyncont
View man page for Sput, making note of the -c parameter
% Sman Sput
% Smkcont cont1
% Sput -c cont1 foo
% Sls
Change directory to the vault, find and cat file
% Slscont
% Slscont -l
% Srmcont
(error, not empty)
% Srm foo
% Srmtrash
% Srmcont cont1
% Slscont
% Sls
Bulk load a whole directory, use time to compare speed
% Sbload -c myContainer x y
Bulk download whole collection
% Sget -b y
Compare speed with Sget -b -y
% Sget -r y
