How To SRB Enable

From SRB

The following is a brief discussion of the various ways one can make use of SRB as part of other applications.

Move or Replicate data to local space. This is the most common option used. You move or replicate or download the input data from SRB space to local space before you start. Once you are done, you move the output back to SRB Space. If the local environment where your app will run can additionally run a SRB server, you replicate the data into this SRB server. The physical name (local FS name) of the file can be obtained from SRB and passed on to the application. If you set up a GridFTP-SRB gateway server, you could also run GridFTP commands to move the data to your local host. Similarly, an OpenDAP interface could be used.

Modify your Application to make SRB client calls. If you have the source code (C, Java, Perl, or Python), you may be able to change the file access portion of your application to make SRB client library calls instead of unix I/O calls. You may want to add some logic so that you app can call either set, based on user input (users might input something like "file://" or "srb://" to differentiate). When using the SRB logical namespace (which is location independent) the application also becomes location independent. This also makes your application "data grid enabled", with access to the advanced features of SRB. In addition to applications that are written in 'C', Java, Python, or Perl, can use the associated load libraries or classes.

Modify your Application to use the UnixIO or srbio libraries. This is similar to the above item, but in some cases will simplify the task. UnixIO is a simple interface library for applications to access SRB functionality at the object (file) level via calls similar to unix IO calls(open, read, write, close, etc). For more information, see the top of the srbUio.h file and the srbUio.c file. Or you can also use the srbio library by adding an extra line to include srbio.h after stdio.h in the source code before compiling and linking. It will change all Unix calls to SRB calls, provided the file name starts with 'srb://'. For example, if one uses: fd = fopen ("srb://home/srb.sdsc/Srm.c","r"); then, the SRB Object '/home/srb.sdsc/Srm.c' is opened for read. Any other file name is treated as a Unix file and dealt accordingly. See the contributed software web page for more information.

Use Solaris (and maybe Linux) PRELOAD without recompiling. This is a feature we call "UNIX Transparency". It involves using the PRELOAD feature of the Solaris and Linux platform to trap UNIX I/O system calls and replace them with SRB I/O calls so that regular UNIX commands and applications can be used to access and manipulate data stored in SRB without recompiling. There's a README.transparency file in the release with for more info on this. But this may no longer be working on recent versions of Linux.

Mount SRB as file system. This has some tradeoffs associated with it, but can sometimes be quite effective. Applications do not need to be recompiled or relinked. One is SRBfs based on FUSE and developed by NCHC, one of our collaborators in Taiwan. Another one (although it may no longer be working) is the lufs-srb (an extended version of a Linux Userland File System (LUFS)) developed by BIRN (a large US data grid project). See the contributed software web page for more information on both SRBfs and lufs-srb.

Move the app to the data. If it is possible and your storage resource has lots of CPU cycles, you can move the app to the data and run it using some Sproxy operation. This will take away much control over the app.

It may be possible to use SRB over Wide Area File Services(WAFS). File Systems like NFS are very "chatty". In spite of any increase in bandwidth, their latency and wait times don't have desired output. WAFS are usually network level devices that listen to the TCP/IP and reduce the "chattiness" for some pre-determined protocols. We have not studied if or how SRB can benefit from WAFS. I am not comfortable to provide this as the first option - but if this works - it would be great. The catch is no one has tried any thing like this before and it requires specialized hardware/software from vendors (read as co$t). In addition, how well these will work with data-intensive environments where the WAFS cache is not just sufficient has to determined. You could just use WAFS alone, but you will loose the Global Namespace, Logical resources and Replication/Migration facilities provided by SRB.