Registering Resources

From SRB

Registering Resources into SRB:

Registering resources need several parameters.

    ResourceName   the name you plan to give the resource
    ResourceType   should have been registered earlier 
    ResourceClass  should have been registered earlier 
    Location       should have been registered earlier
    maxObjSizeInResource   the size in bytes of maximum file
			  size accomodated by the reource
		          unused by SRB currently in any practical manner.
			  for user information purposes only.
    DefaultPath    this provides a template using
	           which the SRB will construct a path for storing
			files in the resource.

The DefaultPath template needs some explanation.

An example of a defaultPath template is given below: /home/leepro/GRID/SRB/data/Vault/?USER.?DOMAIN/?SPLITPATH/?PATH?DATANAME.?RANDOM.?TIMESEC

As you can see the path is of two parts. The first part "/home/leepro/GRID/SRB/data/Vault/" can be seen as a constant/static part which defines the directory in which to store the files. This part should be existing in that location and the SRB system should have read/write and execute permissions there. We recommend that no other user or group is given any access to this directory.

The second part given by "?USER.?DOMAIN/?SPLITPATH/?PATH?DATANAME.?RANDOM.?TIMESEC" defines a dynamic part of the path for storing the file. Each variable-symbol defined by "?*" in the path is filled in at the time of storage. For example "?USER" will be replaced by the srbUserName of the ingestor. Similarly "?RANDOM" is replaced by a random number that is generated based on the processid of the process running the server.

The variables that can be used here are defined below: They are of three types:

(system-generated)
?RANDOM      - a random number
?TIMESEC     - numeric value  given by time(NULL)
?PROCESSID   - the pid of the process running the server
?DATE        - of the form YYYY/DDD  where DDD is day in the year.
?TIME        - value given by get_time(timeval)
?SPLITPATH   - is slightly complicated. It is replaced by two two-digit
		random numbers separated by a '/' (eg. 34/65). 
	       	This helps to make sure that files are distributed over 
               	a tree of sub-directories instead of being concentarated in
	       	one single-directory. 

(srb-system oriented)
?USER        - data creator srbUserName (eg. sekar)
?OWNER       - data creator (eg. sekar) (same as ?USER)
?DOMAIN      - userDomainName (eg. sdsc)
?DATANAME    - dataName (eg. foo.dat) of the object that is being ingested
?COLLECTION  - collectionName in which the object is being ingested. 
		Note that the full collectionName will be used
		here (eg. /home/sekar.sdsc/MyColl7/MySubColl2)
?DIRCHOP=nnn - nnn is an integer. chop off the first nnn entries of the input
		SRB path (COLLECTION/DATANAME) and graft the rest to the end 
		of the data path. 

(user-given)
?PATH        - a string given by user 
?SHADOW      - a string given by user 
?NN          - a string given by user. NN is a two digit number and hence
		this can have hundred variables in the defaultPath.

The values of (system-generated) and (srb-system oriented) parameters are generated at run time by the SRB server. The (user-given) values are extracted from the "pathName" argument value that the user passes on in their call.

The value is taken from "pathName" in the API call:

     extern intsrbObjCreate(srbConn* conn, int catType, char *objID,
	char *dataTypeName, char *resourceName, char *collectionName, 
	char *pathName, srb_long_t dataSize);

The value is taken from destPathname in the Scommand call:

    Sput [-D  dataType] [-c container] [-S destResourceName] [-P destPathname]  [-R retries] [-fpravsmMV] localFile|localDir destObj

The pathName (or destPathname) can have one of two values types:

(a) a full path which has a "/" as its first character this full path name (eg. /usr/sekar/srb/SRB2.1/MCAT/util/foo.dat) will be used as the path name for creating the file. Note that this should be read/writable by SRB and can be outside the default SRBVault.

(b) a set of name-value pairs that are used to generate the user-given parameters. The name-value pairs are of the form &NAME=VALUE. eg. &SHADOW=alpha&PATH=ab/cd&01=foo&03=bar

In the above case alpha is substituted for ?SHADOW and ab/cd for ?PATH and foo for ?01 and bar for ?03 in the DefaultPath For example if the defaultPath of the resource is /data/SRB/?USER.?DOMAIN/?PATH/?DATANAME and pathname given in srbObjCreate is &PATH=ab/cd then the actual path where the data will be stored is /data/SRB/sekar.sdsc/ab/cd/foo.txt assuming user is sekar@sdsc and the dataname is foo.txt

If any of the variables dont have a counterpart in the user-given values then they are substituted with empty string and also if a user has given other values they are ignored when building the full path name.

For example if the defaultPath of the resource is /data/SRB/?USER.?DOMAIN/?01/?02/?03/?DATANAME.?04 and pathname given in srbObjCreate is &PATH=ab/cd&01=aaaa&03=bbbb&04=ver1&05=5555 then the actual path where the data will be stored is /data/SRB/sekar.sdsc/aaaa//bbbb/foo.txt.ver1

Finally note that we are dealing with strings in both defaultPath when creating reasources and in pathName when creating objects. Hence, any blanks or other characters which are in there can cause problem. Note also so that & should not be used in any place in the given pathName and similarly & should not be used anywhere else in defaultPath. These can create problems.