From SRB

Revision as of 22:15, 15 February 2006; view current revision
←Older revision | Newer revision→


General Information

What does the SRB do?

As the name implies the Storage Resource Broker, brokers storage resources (sorry, couldn't resist). It provides access, via a uniform API, to various types of data storage across local and wide-area networks, and maintains meta-data (data about the data) about each stored object (files). SRB, in conjunction with MCAT, provides a means for accessing data objects and resources through querying their attributes instead of knowing their physical names and/or locations.

The SDSC Storage Resource Broker (SRB) provides the abstraction mechanisms needed to implement data grids, digital libraries, and persistent archives for data sharing, data publication, and data preservation.

Many people, using only a subset of the features, find that using the SRB as global file system is its most compelling function. Users of multiple distributed computing systems find it to be an essential tool to easily and quickly access files from various locations. With the SRB's parallel I/O capabilities, the SRB will transfer files at least as quickly as any other mechanism, and usually faster.

What is zoneSRB? or How can I federate SRB systems?

ZoneSRB (or federated MCAT SRB) is the next generation of the SRB released as Version 3.0. It provides facilties for two or more independent SRB systems to interact with each other and allow for seamless access of data and metadata across these SRB systems. These systems are called the 'zones. More information about zoneSRB can be found at: FedMcat



What kinds of resources does the SRB support?

>Storage resources can be directories in Unix file systems, directories in Windows file systems, archival storage systems such as HPSS (and, previously, UniTree and DMF), binary large objects stored in a DBMS (DB2, Oracle, Illustra), database SQL-queriable objects in DB2 or Oracle, and tape library systems. Tape systems can be combined with disk cache into Compound Resources, and the SRB can function as a complete basic archival storage system. SRB includes a set of tape I/O functions and an interface to the STK tape library system.

Is the SRB Open Source?

No, not exactly, although the full source code is readily available to academic organizations and government agencies. The normal distribution is via source. The client sources, that is the Scommands, client C library, inQ, and Jargon, are freely available via a BSD license. Interfaces on top of these, Matrix for example, are also readily available. For commercial applications, please contact the UCSD Technology Transfer & Intellectual Property Services at invent@ucsd.edu. See <a href=http://www.sdsc.edu/srb/srbOpenSource.html> http://www.sdsc.edu/srb/srbOpenSource.html </a> for more information.

How does the SRB compare to commercial software?

As far as we know, there is no commercial product much like the SRB (except for the commercial version of the SRB, see below). The biggest difference between commercial software and research products like the SRB is the lack of a Quality Assurance testing group. But we do a lot of testing of new features, as do our collaborative sites, and we have recently developed some automatic testing scripts and systems. SRB is also a mature product as it has been in production use since 1997. In 2000, a government agency thoroughly examined the code and provided us with fixes (memory overruns, etc). The design is such that most problems are fail-safe, due the the client/server design, and cross-checks within our MCAT library and of the DMBS systems themselves. We build on the quality and robust-features of modern DBMSs.

Is there a commercial version?

Yes, General Atomics has certain exclusive commercial rights. See http://www.nirvanastorage.com. Nirvana Storage's SRB software split from SDSC SRB software in version 1.1.8 in 2001. Since then SDSC has released multiple versions of the SRB software with substantial enhancements including Zones, Bulk Operations, improved administration/installation, stronger security, extended clients such as inQ, JARGON, MySRB, Matrix and Kepler, and extended metadata capabilities.

How does the SRB relate to Grid technologies?

In many ways:

a) The SRB is a complete data Grid system in itself, and has been since SRB 1.0 in 1997. It operates, in production, as collections of client and server hosts distributed across local and/or wide-area networks, cooperating to provide transparent access to storage resources, data, and meta-data (data about data).

b) We are participating in many data grid research and development / production collaborations, including PPDG, GriPhyN, BaBar, CDL, NASA Information Power Grid and many more. The SRB is either used in production or is being evaluated across multiple projects at NSF, NASA, DOE, DOD, NIH, NLM, NARA, and the Library of Congress. See <a href=http://www.sdsc.edu/srb/Projects/main.html> http://www.sdsc.edu/srb/Projects/main.html</a>.

c) We support the Globus Grid Security Infrastructure (GSI) as an optional method of authentication.
d) The <A HREF="#T9">SDSC Matrix</A> workflow management system is a grid-based system and uses a Web Service Definition Language (WSDL) interface.
e) We plan to develop an OGSA-compliant SRB.

Is the SRB middle-ware?

Yes and no. It can be considered middle-ware like other grid technologies as it can be combined with higher level software and can interoperate with other grid components. But it is also a complete solution itself and does not require other software to be a functional whole, except for a DBMS for the metadata catalog.

How secure is the SRB?

The SRB is quite secure. No computer system is perfectly secure, but the SRB provides a reasonable level of security while still providing convenience features and high performance. The Encrypt1 challenge/response is secure against network eavesdropping, while the use of user passwords is convenient and straight-forward for both users and administrators. Placing user passwords into files on host systems is a convenience, although if a host is compromised, those files could be read and the user's SRB identity assumed. GSI is also secure against network eavesdropping and somewhat less vulnerable against compromised hosts as only temporary delegation certificates are stored in files.

Generally, the SRB is as secure as the DBMS used to store the MCAT and the physical resources used to store the data. User identity is as secure as the client host system.

Since the SRB server runs as a non-root user, it does not present a vulnerability to the OS if compromised. This is a big advantage over software systems that need to be run as root.

Starting in April 2004, SRB releases include a paper describing how to run a secure SRB system: readme.dir/srb-security.html.

Starting with SRB 2.1 (late May, 2003), we provide a mechanism by which SRB data files can be encrypted for both network transmission and storage. This system provides security against network eavesdropping for the data objects exchanged via the SRB and also improves the security of the data objects as they reside on any of the various physical resources. This system was implemented to be efficient, although the encryption and decryption is always a compute-intensive operation and there will be some unavoidable performance penalty. See <a href=http://www.sdsc.edu/srb/SecureAndOrCompressedData.html> http://www.sdsc.edu/srb/SecureAndOrCompressedData.html</a> for more information.

How fast is the SRB?

For transferring large files, SRB will normally be significantly faster than FTP, SCP, or NFS and the like, because of the SRB's parallel I/O capabilities (multiple threads each sending a data stream on the network). Sreplicate and Scp use parallel I/O for large-file data transfers by default, and you can use the -m option on Sput and Sget to select parallel I/O.

For small files, transfers can be a little slower due to the additional interaction with the MCAT (especially to a remote MCAT), but you can use Containers and/or Sbload (Bulk load) and Sbunload (Bulk unload) to greatly speed these. For more information see, the man pages and Container questions in this FAQ. We are working toward enhancing Sget and Sput to do bulk operations to non-container files to speed them up too.

How was the SRB developed?

We were funded through a series of research/development proposals. After the initial version, we used the SRB as a basis for additional proposals and applied research projects. Because of this, the SRB is very customer driven, as we strive to meet the specific needs of current projects. Since the projects are similar in nature (at the SRB level), they often share a similar set of requirements, and we can usually leverage the development for one project to assist in other current and/or future projects. In this sense, the SRB provides a uniform data management fabric layer to build large applications.

How many people work on the SRB?

The SDSC SRB group is currently 11 people. We also encourage others to develop and share new features. We have integrated a number of capabilities developed outside the core group. We receive a lot of advice and suggestions from the community, both formally and not.

What support is provided?

We will freely provide answers and provide some limited support to help get sites up and running with the SRB. There is now a <a href=https://lists.sdsc.edu/mailman/listinfo.cgi/srb-chat> srb-chat email list</a> for SRB admins, developers and users to discuss questions, problems, and solutions (it includes an archive of previous posts). <a href=http://www.sdsc.edu/srb>Our web site</a> includes information on current bugs, future plans, current projects, etc. The SRB tar release contains many README files to explain installation and operation. Of course, many of our activities are collaborative funded projects which include specific development tasks and more extensive support.

What operating systems does the SRB run on?

SRB has been ported to

       a variety of Unix

platforms including Linux, Mac OS X, AIX (ex. SP-2 machines), Solaris, SunOS, SGI Irix

       and to Windows.
       The Windows version of the Server cannot
       be configured with an MCAT (so it talks to one that is), but can
       store and retrieve data from the Windows file system.
       SRB is easily portable to Unix-type OSes.

What authentication mechanisms are available for SRB?

> SRB supports three types of authentication: 1) A basic password-based authentication, 2) password-based authentication in which the password

       is used in a challenge-response protocol so no plain-text password
       is sent on the network ("encrypt1"), and 3) GSI authentication.
       Encrypt1 is a simple and secure stand-alone authentication system.

In both password-based systems, user passwords are stored in the MCAT and users can record their passwords into their ~/.srb/.MdasAuth file to provide convenient and reasonably-secure access.

       <a href=http://www.globus.org/security/>GSI (Globus Grid 
       Security Infrastructure)</a> is convenient when using other Globus
       tools but requires users to acquire Certificates (i.e. a Public
       Key Infrastructure is needed).
       Previously we also supported SEA authentication (SDSC
       Encryption and Authentication system) but now GSI provides similar

What are the future plans for the SRB?

We'd like to see the use of the SRB continue to expand, and expect that it will. We will continue to add new features. We are currently participating in many collaborative projects under various funding agencies, and have every reason to believe that this will continue long term. See our plans for the near future in our <a href=http://srb.npaci.edu/bugzilla/> Bugzilla system</a> which we use to track bugs and pending features.

Where can I find more information about SRB and related systems?

We maintain a set of

       web-pages at <a href=http://www.sdsc.edu/srb>http://www.sdsc.edu/srb</a>

where a lot information about the SRB is available. This FAQ also

       contains many links to additional information on specific topics.  
       There are also many documents included with the release under
       the MCAT and readme.dir directories.

Some general information is also available in <A HREF=http://en.wikipedia.org/wiki/Storage_resource_broker> wikipedia</A>. <p>

What kind of query capabilibities are available?

In SRB, we have two types of metadata: "system" and "user-defined". "system" metadata are maintained by the SRB server and "user-defined" are are maintained by users. We have well defined API's and utilities for querying both "system" and "user-defined" metadata.

How can I SRB-enable my application?

See <a href=http://www.sdsc.edu/srb/howToSrbEnable.html> http://www.sdsc.edu/srb/howToSrbEnable.html</a> for a list of options and links to additional information.

How does SRB compare with competing products?

The following is a list of products that compete with SRB in some manner, with a brief description of some differences. SRB is an integrated solution for distributed data management and distributed shared collections.

  • AFS - SRB is more than a file system. SRB manages metadata, replication, parallel I/O, containers.
  • Veritas - SRB is more than a backup system, it supports replicas, versions, and snapshots of files and containers.

  • Oracle 10g IFS - SRB more than a database system, it supports execution of SQL commands at remote databases.

  • HPSS - SRB is more than an archive system, it integrates access across administrative domains and across heterogeneous storage systems.

  • Avaki - SRB is more than a persistent object environment, it manages relational metadata.

  • Globus toolkit - SRB is more than a set of services. SRB manages consistent state information in distributed environment.