FAQ

From SRB

Revision as of 23:05, 16 February 2006 Wayne (Talk | contribs) How many people work on the SRB? ← Previous diff		Current revision Ptooby (Talk | contribs)
Line 2:		Line 2:
	=== What does the SRB do? ===		=== What does the SRB do? ===
-	As the name implies the Storage Resource Broker, brokers storage	+	As the name implies the DICE Storage Resource Broker, brokers storage
	resources (sorry, couldn't resist). It provides access, via a uniform		resources (sorry, couldn't resist). It provides access, via a uniform
	API, to various types of data storage across local and wide-area		API, to various types of data storage across local and wide-area
Line 10:		Line 10:
	attributes instead of knowing their physical names and/or locations.		attributes instead of knowing their physical names and/or locations.
-	The SDSC Storage Resource Broker (SRB) provides the abstraction	+	The DICE Storage Resource Broker (SRB) provides the abstraction
	mechanisms needed to implement data grids, digital libraries, and		mechanisms needed to implement data grids, digital libraries, and
	persistent archives for data sharing, data publication, and data		persistent archives for data sharing, data publication, and data
-	preservation.	+	preservation.
	Many people, using only a subset of the features, find that		Many people, using only a subset of the features, find that
Line 22:		Line 22:
	as any other mechanism, and usually faster.		as any other mechanism, and usually faster.
-	===What is zoneSRB? or How can I federate SRB systems?===	+	===What is [[Zones|zoneSRB]]? or How can I federate SRB systems?===
-	ZoneSRB (or federated MCAT SRB) is the next generation of the SRB	+	ZoneSRB (or federated MCAT SRB) is the next generation of the SRB
-	released as Version 3.0. It provides facilties for two or more independent	+	released as Version 3.0. It provides facilties for two or more independent
-	SRB systems to interact with each other and allow for seamless access of data	+	SRB systems to interact with each other and allow for seamless access of data
	and metadata across these SRB systems. These systems are called the 'zones.		and metadata across these SRB systems. These systems are called the 'zones.
-	More information about zoneSRB can be found at: [[FedMcat]] and [[README.zones]].	+	More information about zoneSRB can be found at [[Fed MCAT]].
	===What kinds of resources does the SRB support?===		===What kinds of resources does the SRB support?===
	Storage resources can be directories in Unix file systems,		Storage resources can be directories in Unix file systems,
-	directories in Windows file systems,	+	directories in Windows file systems,
	archival storage systems such as HPSS (and, previously,		archival storage systems such as HPSS (and, previously,
-	UniTree and DMF), binary large objects stored in a DBMS (DB2, Oracle,	+	UniTree and DMF), binary large objects stored in a DBMS (DB2, Oracle,
-	Illustra), database SQL-queriable objects in DB2 or Oracle,	+	Illustra), database SQL-queriable objects in DB2 or Oracle,
	and tape library systems. Tape systems can be combined with disk		and tape library systems. Tape systems can be combined with disk
	cache into Compound Resources, and the SRB can function as a complete		cache into Compound Resources, and the SRB can function as a complete
Line 41:		Line 41:
	SRB includes a set of tape I/O functions and an interface to		SRB includes a set of tape I/O functions and an interface to
	the STK tape library system.		the STK tape library system.
-
	===Is the SRB Open Source?===		===Is the SRB Open Source?===
-	No, not exactly, although the full	+	No, not exactly, although the full
	source code is readily available to		source code is readily available to
	academic organizations and government agencies. The normal		academic organizations and government agencies. The normal
-	distribution is via source.	+	distribution is via source.
-	The client sources, that is the Scommands, client C library, inQ, and	+	The client sources, that is the Scommands, client C library, inQ, and
-	Jargon, are freely available via a BSD license. Interfaces on	+	Jargon, are freely available via [[Client_License|a BSD license]]. Interfaces on
	top of these, Matrix for example, are also readily available.		top of these, Matrix for example, are also readily available.
	For commercial applications, please		For commercial applications, please
	contact the UCSD Technology Transfer & Intellectual Property Services		contact the UCSD Technology Transfer & Intellectual Property Services
-	at invent@ucsd.edu. See [Is_SRB_Open_Source]	+	at invent@ucsd.edu. See [[Is_SRB_Open_Source]]
-	</a> for more information.	+	for more information.
	===How does the SRB compare to commercial software?===		===How does the SRB compare to commercial software?===
Line 63:		Line 62:
	products like the SRB is the lack of a Quality Assurance testing		products like the SRB is the lack of a Quality Assurance testing
	group. But we do a lot of testing of new features, as do our		group. But we do a lot of testing of new features, as do our
-	collaborative sites, and we have recently developed some	+	collaborative sites, and we have recently developed some
	automatic testing scripts and systems.		automatic testing scripts and systems.
	SRB is also a mature product as it has been in		SRB is also a mature product as it has been in
Line 72:		Line 71:
	library and of the DMBS systems themselves. We build on the quality		library and of the DMBS systems themselves. We build on the quality
	and robust-features of modern DBMSs.		and robust-features of modern DBMSs.
-
	===Is there a commercial version?===		===Is there a commercial version?===
	Yes, General Atomics has certain exclusive commercial rights. See		Yes, General Atomics has certain exclusive commercial rights. See
	http://www.nirvanastorage.com. Nirvana Storage's SRB software split		http://www.nirvanastorage.com. Nirvana Storage's SRB software split
-	from SDSC SRB software in version 1.1.8 in 2001. Since then SDSC has	+	from DICE SRB software in version 1.1.8 in 2001.
-	released multiple versions of the SRB software with substantial	+
		+	Since then the DICE group has released multiple versions of the SRB software with substantial
	enhancements including Zones, Bulk Operations, improved		enhancements including Zones, Bulk Operations, improved
	administration/installation, stronger security, extended clients such		administration/installation, stronger security, extended clients such
Line 84:		Line 83:
	capabilities.		capabilities.
		+	The SRB protocol used in the DICE SRB system provides many new features and functionalities and differs from the protocol used by Nirvana.
	===How does the SRB relate to Grid technologies?===		===How does the SRB relate to Grid technologies?===
Line 97:		Line 97:
	b) We are participating in many data grid research and development /		b) We are participating in many data grid research and development /
	production collaborations, including PPDG, GriPhyN, BaBar, CDL, NASA		production collaborations, including PPDG, GriPhyN, BaBar, CDL, NASA
-	Information Power Grid and many more.	+	Information Power Grid and many more.
	The SRB is either used in production or is being		The SRB is either used in production or is being
	evaluated across multiple projects at NSF, NASA, DOE, DOD, NIH, NLM,		evaluated across multiple projects at NSF, NASA, DOE, DOD, NIH, NLM,
	NARA, and the Library of Congress.		NARA, and the Library of Congress.
	See		See
-	[SRB_Projects]	+	[[Projects]]
	c) We support the Globus Grid Security Infrastructure (GSI) as an		c) We support the Globus Grid Security Infrastructure (GSI) as an
	optional method of authentication.		optional method of authentication.
	<br>		<br>
-	d) The [[SDSC Matrix]] workflow management system is	+	d) The [[Matrix|DICE Matrix]] workflow management system is
	a grid-based system and uses a Web Service Definition Language (WSDL)		a grid-based system and uses a Web Service Definition Language (WSDL)
	interface.		interface.
Line 114:		Line 114:
	===Is the SRB middle-ware?===		===Is the SRB middle-ware?===
-	Yes and no. It can be considered middle-ware like other grid	+
-	technologies as it can be combined with higher level software and can	+	Yes and no. It can be considered middle-ware like other grid technologies as it can be combined with higher level software, can interoperate with other grid components, and depends on other lower-level systems (archives, file systems, networks, a DBMS for the [[MCAT|metadata catalog]], etc). But unlike many other middle-ware components, SRB does not have to be combined with higher-level software to be utilized. With the various clients (Scommands, inQ, mySRB, etc), SRB is also a complete solution itself, a functional whole.
-	interoperate with other grid components. But it is also a complete	+
-	solution itself and does not require other software to be a functional whole,	+
-	except for a DBMS for	+
-	the metadata catalog.	+
	===How secure is the SRB?===		===How secure is the SRB?===
-	The SRB is quite secure.	+	The SRB is quite secure. No computer system is perfectly secure, but the SRB provides a reasonable level of security while still providing convenience features and high performance. The Encrypt1 challenge/response is secure against network eavesdropping, while the use of user passwords is convenient and straight-forward for both users and administrators. Placing user passwords into files on host systems is a convenience, although if a host is compromised, those files could be read and the user's SRB identity assumed. GSI is also secure against network eavesdropping and somewhat less vulnerable against compromised hosts as only temporary delegation certificates are stored in files.
-	No computer system is perfectly secure,	+
-	but the SRB provides a reasonable level of security while	+
-	still providing convenience features and high performance.	+
-	The Encrypt1 challenge/response is secure	+
-	against network eavesdropping, while the use of user passwords is	+
-	convenient and straight-forward for both users and administrators.	+
-	Placing user passwords into files on	+
-	host systems is a convenience, although if a host is compromised,	+
-	those files could be read and the user's SRB identity assumed.	+
-	GSI is also secure against network	+
-	eavesdropping and somewhat less vulnerable against compromised hosts as	+
-	only temporary delegation certificates are stored in files.	+
-	Generally,	+	Generally, the SRB is as secure as the DBMS used to store the MCAT and the physical resources used to store the data. User identity is as secure as the client host system.
-	the SRB is as secure as the DBMS used to store the MCAT and the	+
-	physical resources used to store the data. User identity is as secure	+
-	as the client host system.	+
-	Since the SRB server runs as a non-root user, it does not present a	+	Since the SRB server runs as a non-root user, it does not present a vulnerability to the OS if compromised. This is a big advantage over software systems that need to be run as root.
-	vulnerability to the OS if compromised. This is a big advantage over	+
-	software systems that need to be run as root.	+
-	Starting in April 2004, SRB releases include a paper describing	+	Starting in April 2004, SRB releases include a paper describing how to run a secure SRB system: readme.dir/srb-security.html.
-	how to run a secure SRB system: readme.dir/srb-security.html.	+
-		+
-	Starting with SRB 2.1 (late May, 2003), we provide a mechanism by which	+
-	SRB data files can be encrypted for both network transmission and storage.	+
-	This system provides security against network eavesdropping for	+
-	the data objects exchanged via the SRB and also improves the security	+
-	of the data objects as they reside on any of the various physical resources.	+
-	This system was implemented	+
-	to be efficient, although the encryption and decryption is always a	+
-	compute-intensive operation and there will be some unavoidable performance	+
-	penalty.	+
-	See [[SecureAndOrCompressedData]]	+
-	for more information.	+
		+	Starting with SRB 2.1 (late May, 2003), we provide a mechanism by which SRB data files can be encrypted for both network transmission and storage. This system provides security against network eavesdropping for the data objects exchanged via the SRB and also improves the security of the data objects as they reside on any of the various physical resources. This system was implemented to be efficient, although the encryption and decryption is always a compute-intensive operation and there will be some unavoidable performance penalty.
		+	See [[Secure Compressed Data]] for more information.
	===How fast is the SRB?===		===How fast is the SRB?===
-	For transferring large files, SRB will normally be significantly	+	For transferring large files, SRB will normally be significantly faster than FTP, SCP, or NFS and the like, because of the SRB's parallel I/O capabilities (multiple threads each sending a data stream on the network). [[Sreplicate]] and [[Scp]] use parallel I/O for large-file data transfers by default, and you can use the -m option on [[Sput]] and [[Sget]] to select parallel I/O.
-	faster than FTP, SCP, or NFS and the like, because of the SRB's	+
-	parallel I/O capabilities (multiple threads each sending a data stream	+
-	on the network). Sreplicate and Scp use parallel I/O for large-file	+
-	data transfers by default, and you can use the -m option on Sput and	+
-	Sget to select parallel I/O.	+
-		+
-	For small files, transfers can be a little slower due to the	+
-	additional interaction with the MCAT (especially to a remote MCAT),	+
-	but you can use Containers and/or Sbload (Bulk load) and Sbunload	+
-	(Bulk unload) to greatly speed these. For more information see, the	+
-	man pages and Container questions in this FAQ. We are working toward	+
-	enhancing Sget and Sput to do bulk operations to non-container files	+
-	to speed them up too.	+
		+	For small files, transfers can be a little slower due to the additional interaction with the MCAT (especially to a remote MCAT), but you can use Containers and/or [[Sbload]] (Bulk load) and [[Sbunload]] (Bulk unload) to greatly speed these. For more information see, the [[Scommand_Manpages|man pages]] and [[Containers]] questions in this FAQ. We are working toward enhancing Sget and Sput to do bulk operations to non-container files to speed them up too.
	===How was the SRB developed?===		===How was the SRB developed?===
-	We were funded through a series of research/development proposals.	+
-	After the initial version, we used the SRB as a basis for additional	+	We were funded through a series of research/development proposals. After the initial version, we used the SRB as a basis for additional proposals and applied research projects. Because of this, the SRB is very customer driven, as we strive to meet the specific needs of current projects. Since the projects are similar in nature (at the SRB level), they often share a similar set of requirements, and we can usually leverage the development for one project to assist in other current and/or future projects. In this sense, the SRB provides a uniform data management fabric layer to build large applications.
-	proposals and applied research projects. Because of this, the SRB is	+
-	very customer driven, as we strive to meet the specific needs of	+
-	current projects. Since the projects are similar in nature (at the SRB	+
-	level), they often share a similar set of requirements, and we can	+
-	usually leverage the development for one project to assist in other	+
-	current and/or future projects. In this sense, the SRB provides a	+
-	uniform data management fabric layer to build large applications.	+
	===How many people work on the SRB?===		===How many people work on the SRB?===
-	The SDSC [[SRB Team]] is currently about 11 people (depending how you count).	+
-	We also encourage others	+	The DICE [[SRB Team]] is currently about 11 people (depending how you count). We also encourage others to develop and share new features. We have integrated a number of capabilities developed outside the core group. We receive a lot of advice and suggestions from the community, both formally and not.
-	to develop and share new features. We have integrated a number	+
-	of capabilities developed outside the core group. We receive a lot of	+
-	advice and suggestions from the community, both formally and not.	+
	===What support is provided?===		===What support is provided?===
-	We will freely provide answers and provide some limited support to	+	We will freely provide answers and provide some limited support to help get sites up and running with the SRB. There is a [https://lists.sdsc.edu/mailman/listinfo.cgi/srb-chat srb-chat email list] for SRB admins, developers and users to discuss questions, problems, and solutions (it includes an archive of previous posts). [http://www.sdsc.edu/srb Our web site] includes information on current bugs, future plans, current projects, etc. The SRB tar release contains many README files to explain installation and operation. Of course, many of our activities are collaborative funded projects which include specific development tasks and more extensive support.
-	help get sites up and running with the SRB. There is a	+
-	[https://lists.sdsc.edu/mailman/listinfo.cgi/srb-chat srb-chat email list]	+
-	for SRB admins, developers and users to discuss	+
-	questions, problems, and solutions (it includes an archive of previous posts).	+
-	[http://www.sdsc.edu/srb Our web site]	+
-	includes information	+
-	on current bugs, future plans, current projects, etc. The SRB tar	+
-	release contains many README files to explain installation and	+
-	operation. Of course, many of our activities are collaborative funded	+
-	projects which include specific development tasks and more extensive	+
-	support.	+
	===What operating systems does the SRB run on?===		===What operating systems does the SRB run on?===
-	SRB has been ported to a variety of Unix platforms including Linux,	+	SRB has been ported to a variety of Unix platforms including Linux, Mac OS X, AIX (ex. SP-2 machines), Solaris, SunOS, SGI Irix and to Windows. The Windows version of the Server cannot be configured with an MCAT (so it talks to one that is), but can store and retrieve data from the Windows file system. SRB is easily portable to Unix-type OSes.
-	Mac OS X, AIX (ex. SP-2 machines), Solaris, SunOS, SGI Irix and to	+
-	Windows. The Windows version of the Server cannot be configured with	+
-	an MCAT (so it talks to one that is), but can store and retrieve data	+
-	from the Windows file system. SRB is easily portable to Unix-type	+
-	OSes.	+
-		+
	===What authentication mechanisms are available for SRB?===		===What authentication mechanisms are available for SRB?===
	SRB supports three types of authentication: 1) A basic password-based		SRB supports three types of authentication: 1) A basic password-based
-	authentication, 2) password-based authentication in which the password	+	authentication (now phased out), 2) password-based authentication in which the password
	is used in a challenge-response protocol so no plain-text password is		is used in a challenge-response protocol so no plain-text password is
-	sent on the network ("encrypt1"), and 3) GSI authentication. Encrypt1	+	sent on the network ("[[Encrypt1]]"), and 3) GSI authentication. Encrypt1
	is a simple and secure stand-alone authentication system. In both		is a simple and secure stand-alone authentication system. In both
	password-based systems, user passwords are stored in the MCAT and		password-based systems, user passwords are stored in the MCAT and
	users can record their passwords into their ~/.srb/.MdasAuth file to		users can record their passwords into their ~/.srb/.MdasAuth file to
-	provide convenient and reasonably-secure access. <a	+	provide convenient and reasonably-secure access. [http://www.globus.org/security GSI] (Globus Grid Security
-	href=http://www.globus.org/security/>GSI (Globus Grid Security	+	Infrastructure) is convenient when using other Globus tools but
-	Infrastructure)</a> is convenient when using other Globus tools but	+
	requires users to acquire Certificates (i.e. a Public Key		requires users to acquire Certificates (i.e. a Public Key
	Infrastructure is needed). Previously we also supported SEA		Infrastructure is needed). Previously we also supported SEA
	authentication (SDSC Encryption and Authentication system) but now GSI		authentication (SDSC Encryption and Authentication system) but now GSI
	provides similar functionality.		provides similar functionality.
-
-
	===What are the future plans for the SRB?===		===What are the future plans for the SRB?===
Line 248:		Line 174:
	participating in many collaborative projects under various funding		participating in many collaborative projects under various funding
	agencies, and have every reason to believe that this will continue		agencies, and have every reason to believe that this will continue
-	long term. See our plans for the near future in our	+	long term. See our plans for the near future in our
	[Bugzilla] system which we use to track bugs and pending features.		[Bugzilla] system which we use to track bugs and pending features.
-
	===Where can I find more information about SRB and related systems?===		===Where can I find more information about SRB and related systems?===
-	We maintain a set of	+	We maintain a set of web-pages at http://www.sdsc.edu/srb where a lot information about the SRB is available. This FAQ also contains many links to additional information on specific topics. There are also many documents included with the release under the MCAT and readme.dir directories.
-	web-pages at <a href=http://www.sdsc.edu/srb>http://www.sdsc.edu/srb</a>	+
-	where a lot information about the SRB is available. This FAQ also	+
-	contains many links to additional information on specific topics.	+
-	There are also many documents included with the release under	+
-	the MCAT and readme.dir directories.	+
-		+
-	Some general information is also available in	+
-	[http://en.wikipedia.org/wiki/Storage_resource_broker>	+
-	wikipedia].	+
-	or	+
-	*http://en.wikipedia.org/wiki/Storage_resource_broker.	+
		+	Some general information is also available in http://en.wikipedia.org/wiki/Storage_resource_broker.
	===What kind of query capabilibities are available?===		===What kind of query capabilibities are available?===
-	In SRB, we have two types of metadata: "system" and "user-defined".	+	In SRB, we have two types of [[metadata]]: "system" and "user-defined".
	"system" metadata are maintained by the SRB server and "user-defined"		"system" metadata are maintained by the SRB server and "user-defined"
-	are are maintained by users. We have well defined API's and utilities	+	are maintained by users. We have well defined API's and utilities
	for querying both "system" and "user-defined" metadata.		for querying both "system" and "user-defined" metadata.
-
	===How can I SRB-enable my application?===		===How can I SRB-enable my application?===
-	See	+	See [[How To SRB Enable]] for a list of options and links to additional information.
-	[[howToSrbEnable.html]	+
-	for a list of options	+
-	and links to additional information.	+
	===How does SRB compare with competing products?===		===How does SRB compare with competing products?===
Line 289:		Line 200:
	integrated solution for distributed data management and		integrated solution for distributed data management and
	distributed shared collections.		distributed shared collections.
-	<UL>	+	*AFS - SRB is more than a file system. SRB manages metadata, replication, parallel I/O, containers.
-	<LI>	+	*Veritas - SRB is more than a backup system, it supports replicas, versions, and snapshots of files and containers.
-	AFS - SRB is more than a file system.	+	*Oracle 10g IFS - SRB is more than a database system, it supports execution of SQL commands at remote databases.
-	SRB manages metadata, replication, parallel I/O, containers.	+	*HPSS - SRB is more than an archive system, it integrates access across administrative domains and across heterogeneous storage systems.
-	</li><LI>	+	*Avaki - SRB is more than a persistent object environment, it manages relational metadata.
-	Veritas - SRB is more than a backup system, it	+	*Globus toolkit - SRB is more than a set of services. SRB manages consistent state information in distributed environment.
-	supports replicas, versions, and snapshots of files and containers.	+
-	</li><LI>	+
-	Oracle 10g IFS - SRB more than a database system, it	+
-	supports execution of SQL commands at remote databases.	+
-	</li><LI>	+
-	HPSS - SRB is more than an archive system, it	+
-	integrates access across administrative domains and across	+
-	heterogeneous storage systems.	+
-	</li><LI>	+
-	Avaki - SRB is more than a persistent object environment,	+
-	it manages relational metadata.	+
-	</li><LI>	+
-	Globus toolkit - SRB is more than a set of services.	+
-	SRB manages consistent state information in distributed environment.	+
-	</li>	+
-	</UL>	+
	==Interfaces and Tools==		==Interfaces and Tools==
-	===What are the Scommands?===	+	===What are the [[Scommands]]?===
	Scommands refers to a set of utility routines that can be used in a		Scommands refers to a set of utility routines that can be used in a
	Unix shell or Windows DOS command shell and access data and meta data		Unix shell or Windows DOS command shell and access data and meta data
	information from SRB and MCAT.		information from SRB and MCAT.
-	For more information on Scommands see README.utilities. Scommands	+	For more information on Scommands see [[User_Environment]]. Scommands
	also have a set of man pages describing each of the commands.		also have a set of man pages describing each of the commands.
-	One first logs in via a Sinit, and can then do Sls, Scd, Sput, Sget, etc.	+	One first logs in via a [[Sinit]], and can then do [[Sls]], [[Scd]], [[Sput]], [[Sget]], etc.
-	Man pages are available at	+	For the full list see the [[Scommand_Manpages|Scommands Man pages]].
-	[http://nbirn.ucsd.edu/ForUsers/Tutorials/SRB/manpagesv20.html]	+
-	and	+
-	[http://www.sdsc.edu/srb/srbcommands.html]	+
-	.	+
-	===What is inQ?===	+	===What is [[inQ]]?===
	inQ is a graphical SRB client for Windows 98/Me/NT/2k/XP.		inQ is a graphical SRB client for Windows 98/Me/NT/2k/XP.
-	In a nutshell, inQ provides a familiar file-manager-like interface that SRB	+	In a nutshell, inQ provides a familiar file-manager-like interface that SRB users can use to manage their data stored on SRB; actually it's more like a file-manager interface on steroids. inQ looks and acts a lot like Windows Explorer or Nautilus but also throws in features found in several web browsers like Internet Explorer or Netscape Navigator. It offers an easy way to manage metadata and access permissions, as well as a query builder capable of performing nested queries. It also throws in friendly, context-sensitive buttons that show you which actions can be performed on any given item in SRB.
-	users can use to manage their data stored on SRB; actually it's more like a	+
-	file-manager interface on steroids. inQ looks and acts a lot like Windows	+
-	Explorer or Nautilus but also throws in features found in several web	+
-	browsers like Internet Explorer or Netscape Navigator. It offers an easy way	+
-	to manage metadata and access permissions, as well as a query builder	+
-	capable of performing nested queries. It also throws in friendly,	+
-	context-sensitive buttons that show you which actions can be performed on	+
-	any given item in SRB.	+
-	For more	+	===What is [[MySRB]]? ===
-	information, see [inQ].	+
-	===What is MySRB? ===	+	MySRB is a web-based browse and search interface to the SRB. See the mySRB home page at http://www.sdsc.edu/srb/mySRB/mySRB.html for more information.
-		+
-	MySRB is a web-based browse and search interface to the SRB.	+
-	See the mySRB home page at	+
-	<a href=http://www.sdsc.edu/srb/mySRB/mySRB.html>	+
-	http://www.sdsc.edu/srb/mySRB/mySRB.html	+
-	for more information.	+
	===What APIs are available?===		===What APIs are available?===
-	The most comprehensive programmatic API is the SRB	+	The most comprehensive programmatic API is the SRB C library which can be linked with any application program. We also have a pure Java client library, [[Jargon]], which includes all the client function calls. Almost all of the C library calls can be accessed through our Python binding. Some sample programs for using the API can be found in the release under test/examples. Also see the API description at [[C_API]] and [[SRB User Manual]]
-	C library which can be linked with any application program.	+
-	We also have a pure Java client	+
-	library, which contains the most commonly used function calls	+
-	[Jargon]	+
-	Almost all of the	+
-	C library calls can be accessed through our Python binding.	+
-	Some sample programs for using the API can be found in	+
-	the release under test/examples.	+
-	Also see the API description at	+
-	[README.clientAPI]	+
-	http://www.sdsc.edu/srb/install/README.clientAPI</a>	+
-	and	+
-	the SRB Technical Information page at	+
-	<a href=http://www.sdsc.edu/srb/srb.html>	+
-	http://www.sdsc.edu/srb/srb.html</a>.	+
-	<p>	+
	===What is the srbBrowser? ===		===What is the srbBrowser? ===
-	The srbBrowser is a java-based graphical SRB client. It provides a subset	+	The srbBrowser is a java-based graphical SRB client. It provides a subset of the functionality of inQ but can be used as a graphical client on Unix systems.
-	of the functionality of inQ but can be used as a graphical client on Unix	+
-	systems.	+
		+	===What is the [[Admin_Tool|mcatAdmin (Java Admin Tool)]]? ===
-	===What is the mcatAdmin (Java Admin Tool) ? ===	+	mcatAdmin (also commonly called the Java Admin Tool) is a java-based graphical (GUI) srb-mcat Administration tool. It assists in the administration by making clear the available functions (like most GUIs) and presenting available values from which to choose. For example, when adding a new user, the existing domains are listed and the adminstrator clicks on the domain to use for the new user. And when modifying a user, one clicks on a domain and is given a list of the users in that domain to choose from. The GUI includes windows to create, display, and modify zones, users, resources, locations, domains, and other tokens. There are also command-line utilities that perform administrative functions.
-	mcatAdmin (also commonly called the Java Admin Tool) is a java-based	+	===What is [[Jargon]] (Java API)?===
-	graphical (GUI) srb-mcat Administration tool. It assists in the	+
-	administration by making clear the available functions (like most	+
-	GUIs) and presenting available values from which to choose. For	+
-	example, when adding a new user, the existing domains are listed and	+
-	the adminstrator clicks on the domain to use for the new user. And	+
-	when modifying a user, one clicks on a domain and is given a list of	+
-	the users in that domain to choose from. The GUI includes windows to	+
-	create, display, and modify zones, users, resources, locations,	+
-	domains, and other tokens. There are also command-line utilities that	+
-	perform administrative functions.	+
		+	JARGON is a pure Java API for developing SRB (or other) datagrid interfaces. The API currently handles file I/O for local and SRB file systems and is easily extensible to other file systems. File handling with JARGON closely matches file handling in Sun's java.io API, a familiar API to most java programmers.
-	===What is Jargon (Java API)?===	+	===What is the DICE [[Matrix]]?===
-	JARGON is a pure Java API for developing SRB (or other) datagrid interfaces.	+	DICE Matrix is a data grid workflow management system. Matrix can be used to create, access and manage workflow process pipelines. Matrix internally uses the Data Grid Language, which can be used to describe, query and control process-flow pipelines.
-	The API currently handles file I/O for	+
-	local and SRB file systems and is easily extensible to other file	+
-	systems. File handling with JARGON closely matches file handling in	+
-	Sun's java.io API, a familiar API to most java programmers.	+
-	[Jargon]	+
-	===What is the SDSC Matrix?===	+	The Matrix API can be used to define multiple SRB commands (and non-SRB grid services) as a single dataflow process and execute it on multiple servers. Matrix is available as a (SOAP/WSDL) web service. Matrix client programming for SRB is made very simple using a developer friendly Java API (less learning curve).
-		+
-	SDSC Matrix is a data grid workflow management system. Matrix can be used to	+
-	create, access and manage workflow process pipelines. Matrix internally uses	+
-	the Data Grid Language, which can be used to describe, query and control	+
-	process-flow pipelines.	+
-	See [Matrix] for more information.	+
-		+
-	The Matrix API can be used to define multiple SRB commands (and non-SRB grid	+
-	services) as a single dataflow process and execute it on multiple servers.	+
-	Matrix is available as a (SOAP/WSDL) web service. Matrix client programming	+
-	for SRB is made very simple using a developer friendly Java API (less	+
-	learning curve).	+
	===Does MCAT functionality vary from one client to another?===		===Does MCAT functionality vary from one client to another?===
-	All functionalities are supported in the Scommand utilities for	+	All functionalities are supported in the Scommand utilities for Unix/Linux/MacOSX and Windows. This is because we do all development on Unix clients and they get ported to other platforms. The MySRB provides a different perspective to metadata management but on a single-file level and collection level. It provides a good way of browsing and querying of metadata across collections, and also allows for ingesting, extracting, updating and deleting metadata and user annotations for single SRBobject or SRB collection. The inQ provides a unique capability where one can associate metadata to SRB objects and collections in an intuitive way and also query across collection and form (temporary) query-collections. This allows one to query based on attribute-metadata and then get a collection and slowly refine the query to drill down to a sub-collection that is of interest.
-	Unix/Linux/MacOSX and Windows. This is because we do all development on	+
-	Unix clients and they get ported to other platforms. The MySRB provides a	+
-	different perspective to metadata management but on a single-file level	+
-	and collection level. It provides a good way of browsing and querying	+
-	of metadata across collections, and also allows for ingesting,	+
-	extracting, updating and deleting metadata and user annotations for	+
-	single SRBobject or SRB collection. The inQ provides a unique capability	+
-	where one can associate metadata to SRB objects and collections in an	+
-	intuitive way and also query across collection and form (temporary)	+
-	query-collections. This allows one to query based on attribute-metadata	+
-	and then get a collection and slowly refine the query to drill down to	+
-	a sub-collection that is of interest.	+
-	Hence, each client provides a unique way of handling metadata and their	+	Hence, each client provides a unique way of handling metadata and their management. One of our goals is to provide uniform functionality across all client interfaces. But this requires a huge amount of programming which we are unable to dedicate at this time.
-	management. One of our goals is to provide uniform functionality across	+
-	all client interfaces. But this requires a huge amount of programming	+
-	which we are unable to dedicate at this time.	+
	===Is one client better than another for entering metadata?===		===Is one client better than another for entering metadata?===
-	The Scommands client is very good for entering metadata. As mentioned	+	The Scommands client is very good for entering metadata. As mentioned before one can use inQ or MySRB for entering/updating metadata of individual SRB object and SRB collection. But the Scommands provide for Bulk ingestion of metadata for multiple SRB objects possibly residing in more than one SRB collection.
-	before one can use inQ or MySRB for entering/updating metadata of	+
-	individual SRB object and SRB collection. But the Scommands provide	+
-	for Bulk ingestion of metadata for multiple SRB objects possibly	+
-	residing in more than one SRB collection.	+
-	For cutting and pasting, there are utilities in both MySRB and in	+	For cutting and pasting, there are utilities in both MySRB and in Scommands for copying metadata from one SRBobject to another, from one SRB collection to another and from one SRB collection to an SRB object. This is different from cutting and pasting as it is done internal to the SRB and not at the user-GUI.
-	Scommands for copying metadata from one SRBobject to another, from one	+
-	SRB collection to another and from one SRB collection to an SRB	+
-	object. This is different from cutting and pasting as it is done	+
-	internal to the SRB and not at the user-GUI.	+
	===Is there a way to load attribute/value pairs from another application into the MCAT?===		===Is there a way to load attribute/value pairs from another application into the MCAT?===
Line 474:		Line 284:
	or through the Scommands.		or through the Scommands.
-	==MetaData Catalog (MCAT)==	+	==[[MCAT|MetaData Catalog (MCAT)]]==
-	===What is MCAT?===	+	===What is [[MCAT]]?===
-	MCAT, or Meta data Catalog, is a meta data repository system	+	MCAT, or Meta data Catalog, is a meta data repository system
-	implemented at SDSC to provide a mechanism for storing and querying	+	implemented by the DICE group to provide a mechanism for storing and querying
	system-level and domain-dependent meta data using a uniform interface.		system-level and domain-dependent meta data using a uniform interface.
	MCAT provides a resource and data object discovery mechanism that can be		MCAT provides a resource and data object discovery mechanism that can be
Line 486:		Line 296:
	instead of their physical names and/or locations.		instead of their physical names and/or locations.
-	===What is meta data?===	+	===What is [[Metadata|meta data]]?===
-	Meta data is information about data.	+	Meta data is information about data.
-		+	===What is system-level meta data?===
-	===What is system-level meta data?===	+
	MCAT considers five kinds of entities as primitive objects on which it		MCAT considers five kinds of entities as primitive objects on which it
-	keeps additional information. These are: data objects, resources,	+	keeps additional information. These are: '''data objects''', '''resources''',
-	collections, users and methods. The system-level MCAT meta data items	+	'''collections, users''' and''' methods.''' The system-level MCAT meta data items
	are these primitive objects and others derived from these.		are these primitive objects and others derived from these.
-
	===Since there are primitive MCAT objects, are there other MCAT objects?===		===Since there are primitive MCAT objects, are there other MCAT objects?===
Line 503:		Line 311:
	release, supports notions of logical resources, compound resources,		release, supports notions of logical resources, compound resources,
	user groups, etc.		user groups, etc.
-
	===What is application-level meta data?===		===What is application-level meta data?===
Line 519:		Line 326:
	data object such as FITS metadata used in Astronomy and DICOM metadata		data object such as FITS metadata used in Astronomy and DICOM metadata
	for medical images.		for medical images.
-		+
	===What is domain-dependent meta data?===		===What is domain-dependent meta data?===
Line 533:		Line 340:
	===What databases can be used for installing MCAT?===		===What databases can be used for installing MCAT?===
-	MCAT can be installed on either Oracle, DB2, Sybase, Postgres, or	+	MCAT can be installed on either Oracle, DB2, Sybase, Postgres,
-	Informix. SQLServer, since it is so similar to Sybase, should be	+	Informix, or MySQL 4. SQLServer, since it is so similar to Sybase, should be
	fairly straight-forward to implement too.		fairly straight-forward to implement too.
		+
		+	===Can the MCAT run on 64-bit architectures (x86_64)?===
		+
		+	The MCAT code cannot run in 64-bit mode, but if you use the compiler option
		+	to build in 32-bit mode and are using Oracle, the MCAT will work fine on 64-bit hosts.
		+	Using postgresQL there are additional problems in running in this mixed mode
		+	and we do not currently support this.
		+	Non-MCAT servers run on a 64-bit architecture without problem and can be
		+	integrated into an SRB datagrid like any other.
	==Administration/Operation==		==Administration/Operation==
Line 555:		Line 371:
	process.		process.
-		+	===What are the hardware requirements (disk space, CPU speed, memory) for an SRB Server host? ===
-	===What are the hardware requirements (disk space, CPU speed, memory)	+
-	for an SRB Server host? ===	+
	The hard disk size depends upon how much storage you want to		The hard disk size depends upon how much storage you want to
	broker. The SRB software system itself requires only about 200 MBytes		broker. The SRB software system itself requires only about 200 MBytes
-	of storage. For MCAT-enabled servers, the DBMS will require	+	of storage. For MCAT-enabled servers, the DBMS will require
	additional space; on Linux, for example, the SRB with Postgres		additional space; on Linux, for example, the SRB with Postgres
	and ODBC take about 700 MB.		and ODBC take about 700 MB.
Line 571:		Line 385:
	As for CPU speeds, any Linux system with more than 1.5 GHz should be fine.		As for CPU speeds, any Linux system with more than 1.5 GHz should be fine.
	Memory of 1/2 GB or 1 GB will be sufficient.		Memory of 1/2 GB or 1 GB will be sufficient.
-
	===Which DB system should I use?===		===Which DB system should I use?===
Line 610:		Line 423:
	SRB users are registered in the MCAT catalog and are given unique SRB ids.		SRB users are registered in the MCAT catalog and are given unique SRB ids.
-	These identifiers are independent of the location or system ids, such as	+	These identifiers are independent of the location or system ids, such as
	Unix ids.		Unix ids.
-
	===What is a method?===		===What is a method?===
Line 632:		Line 444:
	has additional privileges compared to normal users. A SRBadmin		has additional privileges compared to normal users. A SRBadmin
	does NOT need to have "root" privilege.		does NOT need to have "root" privilege.
-
	===What is a (data object) collection?===		===What is a (data object) collection?===
Line 645:		Line 456:
	a collection can be stored in heterogeneous storage devices. There is		a collection can be stored in heterogeneous storage devices. There is
	one obvious restriction, the name given to a data object in a		one obvious restriction, the name given to a data object in a
-	collection or sub-collection should be unique in that collection. <p>	+	collection or sub-collection should be unique in that collection.
	===What is the SRB logical name space?===		===What is the SRB logical name space?===
Line 663:		Line 474:
	system adds this logical name space on top of the physcial name space,		system adds this logical name space on top of the physcial name space,
	and derives much of its power and functionality from that.		and derives much of its power and functionality from that.
-
	===What is a resource?===		===What is a resource?===
Line 671:		Line 481:
	"physical resource". For example, HPSS can be a resource, as can a		"physical resource". For example, HPSS can be a resource, as can a
	Unix file system.		Unix file system.
-
	===What is a physical SRB resource?===		===What is a physical SRB resource?===
Line 680:		Line 489:
	its physical characteristics such as its physical location, resource		its physical characteristics such as its physical location, resource
	type, latency, and maximum file size.		type, latency, and maximum file size.
-
	===What is a logical SRB resource?===		===What is a logical SRB resource?===
Line 719:		Line 527:
	===What is a user group?===		===What is a user group?===
	A user group is a uniquely identifiable name given to a set		A user group is a uniquely identifiable name given to a set
-	of SRB registered users.	+	of SRB registered users.
	===Who can form a user group?===		===Who can form a user group?===
Line 725:		Line 533:
	===Who can register a user group?===		===Who can register a user group?===
-	SRBadmin has the authority to register user groups.	+	SRBadmin has the authority to register user groups.
	===What is a domain?===		===What is a domain?===
Line 732:		Line 540:
	uniquely identified by their usernames combined with their domain		uniquely identified by their usernames combined with their domain
	'smith@npaci'. SRBadmin has the authority to create domains.		'smith@npaci'. SRBadmin has the authority to create domains.
-
	===What are tokens?===		===What are tokens?===
Line 745:		Line 552:
	different locations. But, all these copies in SRB are considered to be		different locations. But, all these copies in SRB are considered to be
	identifiable by the same identifier. That is, each copy is considered		identifiable by the same identifier. That is, each copy is considered
-	to be equivalent to each other.	+	to be equivalent to each other.
	===How can one read a replicated data object?===		===How can one read a replicated data object?===
Line 751:		Line 558:
	When a user reads a replicated data object, SRB cycles through all the		When a user reads a replicated data object, SRB cycles through all the
	copies of the data object and reads the one that is accessible at that		copies of the data object and reads the one that is accessible at that
-	time. It uses a simple replica identificatoin mechanism to order this	+	time. It uses a simple mechanism to order this list of replicated data objects. Copies stored in the resources belonging to the "cache" class (cache, temporary) will be put at the top of the list and copies stored in the "archival" class (archival, permanant) are put at the bottom of the list. For copies belonging to the same class, the copy stored on the same host as the one directly connected to the client will be picked first. After that, the copies within the same class will be picked randomly.
-	list of replicated data objects.	+
	===How can one create a replicated data object?===		===How can one create a replicated data object?===
-	There are three ways of creating a replicated data object.	+	There are three ways of creating a replicated data object.
	In the first method, which can be viewed as asynchronous replication,		In the first method, which can be viewed as asynchronous replication,
	one can create a data object (using Sput Scommand or srbObjCreate API),		one can create a data object (using Sput Scommand or srbObjCreate API),
-	and then replicate the data object using the Sreplicate Scommand or the	+	and then replicate the data object using the Sreplicate Scommand or the
	srbObjReplicate API. In the second method, which can be viewed as		srbObjReplicate API. In the second method, which can be viewed as
	synchronous replication, one can define a 'logical resource set'		synchronous replication, one can define a 'logical resource set'
Line 771:		Line 577:
	collections of data objects recursively.		collections of data objects recursively.
-	Also see "Replicated Data Management user SRB" (GGF-4, February, 2002) at	+	Also see "Replicated Data Management user SRB" (GGF-4, February, 2002) at http://www.sdsc.edu//srb/Pubs/SRBReplication.ppt.
-	<a href=http://www.npaci.edu/dice/Pubs/SRBReplication.ppt>	+
-	http://www.npaci.edu/dice/Pubs/SRBReplication.ppt</a>.	+
-	===What is a Container?===	+	===What is a [[Container]]?===
	A Container is a way to put together a lot of small files into one		A Container is a way to put together a lot of small files into one
Line 826:		Line 630:
-	===Does Sget work properly for files that are in containers?===	+	===Does [[Sget]] work properly for files that are in containers?===
	Yes, Sget works fine for files in containers. The MCAT stores all		Yes, Sget works fine for files in containers. The MCAT stores all
Line 832:		Line 636:
	just the portion of the container that has the file you are trying to		just the portion of the container that has the file you are trying to
	download. Since Sget (currently) doesn't support any bulk operations		download. Since Sget (currently) doesn't support any bulk operations
-	it's still slow trying to download a lot of small files.	+	it's still slow trying to download a lot of small files.
	===How do you discover the container information?===		===How do you discover the container information?===
	If you're on a windows machine InQ is the easiest, the file details		If you're on a windows machine InQ is the easiest, the file details
-	show the container information. In Scommands, SgetD on a file prints	+	show the container information. In Scommands, [[SgetD]] on a file prints
	container_name and the respective container (if any) that the file is		container_name and the respective container (if any) that the file is
	contained in.		contained in.
Line 857:		Line 661:
	===How does SRB provide access to remote storage systems?===		===How does SRB provide access to remote storage systems?===
	SRB provides access to remote storage systems through a proxy		SRB provides access to remote storage systems through a proxy
-	mechanism. When one stores a data object under SRB, the data object is	+	mechanism. When one stores a data object under SRB, the data object is
-	stored and accessed by SRB acting as a proxy for the user. Because	+	stored and accessed by SRB acting as a proxy for the user. Because
-	of this mechanism, a user can store data objects on remote storage	+	of this mechanism, a user can store data objects on remote storage
-	systems without having personal accounts at these site. In this mode,	+	systems without having personal accounts at these site. In this mode,
-	SRB acts as a 'system privileged proxy' user. The above proxy mode	+	SRB acts as a 'system privileged proxy' user. The above proxy mode
-	also allows for SRB to SRB authentication enabling servers to	+	also allows for SRB to SRB authentication enabling servers to
-	access files that are under the control of another SRB server.	+	access files that are under the control of another SRB server.
	===Can multiple SRB servers be federated?===		===Can multiple SRB servers be federated?===
Line 889:		Line 693:
	(3) server with MCAT.		(3) server with MCAT.
-	Each of these three setups can be enabled with password,	+	Each of these three setups can be enabled with password-[[encrypt1]], and/or [[GSI]] authentication.
-	password-encrypt1, and/or GSI authentication.	+
	In the simplest configuration, one can use the SRB client components		In the simplest configuration, one can use the SRB client components
Line 909:		Line 712:
	===Is MCAT needed to run SRB?===		===Is MCAT needed to run SRB?===
	Yes, an MCAT is needed but you do not need to install one yourself.		Yes, an MCAT is needed but you do not need to install one yourself.
-	Many sites use the SDSC MCAT-enabled SRB to support their SRB system.	+	Many sites use the DICE MCAT-enabled SRB to support their SRB system.
	===Where can I get SRB?===		===Where can I get SRB?===
-	Source code and related material for SRB and MCAT can be obtained	+	Source code and related material for SRB and MCAT can be obtained
-	from the web-site at <a href=http://www.sdsc.edu/srb>	+	from the web-site at http://www.sdsc.edu/srb. The tar files
-	http://www.sdsc.edu/srb</a>. The tar files	+