Srsync

From SRB

Contents

NAME

Srsync - Synchronize the data between a local copy (local file system) and the copy stored in SRB or between two SRB copies.

SYNOPSIS

Srsync [-S resource] [-t tmpInxDir] [-rvamMlLs] sourceFile|sourceDirectory [....] targetFile|targetDirectory


DESCRIPTION

Srsync synchronizes the data between a local copy (local file system) and the copy stored in SRB or between two SRB copies. The command can be in one of the three modes  : synchronization of data from the client's local file system to SRB, from SRB to the local file system, or from one SRB path to another SRB path. The mode is determined by the way the sourceFile|sourceDirectory and targetFile|targetDirectory are specified. Files and directories prepended with "s:" are SRB files and collections. Local files and directories are specified without any prependage.

For example, the command:

 Srsync -r foo1 s:foo2

synchronizes recursively the data from the local directory "foo1" to the SRB collection "foo2" and the command:

 Srsync -r s:foo1 foo2

synchronizes recursively the data from the SRB collection "foo1" to the local directory "foo2".

 Srsync -r s:foo1 s:foo2

synchronizes recursively the data from the SRB collection "foo1" to another SRB collection "foo2". Cross-zone synchronization is allowed.

The command compares the checksum values and file sizes of the source and target files to determine whether synchronization is needed. Therefore, the command will run faster if the checksum value for the specific SRB file, no matter whether it is a source or target, already exists and is registered with MCAT. This can be achieved by using the -k or -K options of the Sput command at the time of ingestion, or by using the Schksum command after the data have already been ingested into SRB.

If the -s option is used, only the file size (instead of the the size and checksum value) is used for determining whether synchronization is needed. This mode is gives a faster operation but the result is less accurate.


The command accepts multiple sourceFiles|sourceDirectories and a single targetFile|targetDirectory. It pretty much follows the syntax of the UNIX "cp" command with one exception: Srsync of a single source directory to a single target directory. In UNIX, the command:

cp -r foo1 foo2

has a different meaning depending on whether the target directory "foo2" exists. If the target directory exists, the content of source directory "foo1" is copied to the target directory "foo2/foo1". But if the target directory does not exist, the content is copied to the target directory "foo2".

With the Srsync command,

 Srsync -r foo1 s:foo2

always means the synchronization of the local directory "foo1" to collection "foo2", no matter whether "foo2" exists or not.

The optional [-t tmpInxDir] option specifies a directory where the temporary index files generated by the Srsync command will be written into. For, efficiency reasons, the Srsync does a single query to get all the metadata (all files in a collection plus the checksum value and file size associated with each file) it needs for the synchronization operation. The result of the query can be huge and may not easily fit in memory. Currently, if the number of files exceeds 2,000, it will start writing the query results in some temporary index files in the directory specified by "tmpInxDir" if the -t option is used. If the -t option is not used by default it will try the directories in the following order: cwd, "/tmp" and finally "~".

The -l option lists all the files that are different between the source and target plus the files that exist in the source but not in the target without actually doing the synchronization.


OPTIONS

-a
synchronize to all replica if the target is a SRB file/collection.
-m
use "server initiated" (see Sput/Sget for explanation) parallel I/O for synchronization.
-M
use "client initiated" (see Sput/Sget for explanation) parallel I/O for synchronization.
-s
use the size instead of the checksum value for determining synchronization.
-l
lists all the files that are different between the source and target without actually doing the synchronization.
-L
Follow symbolic links if the source is in local disk. By default, Srsync will exclude symbolic links.


-r
recursively synchronize files in the directory/collection and subdirectory/subcollections.
-v
verbose mode. Not relevant with the -l option.
-S
resourceName is the target resourceName if the target file does not exist. Not relevant if the target file exists.
-t
tmpInxDir specifies a directory where the temporary index files generated by the Srsync command will be written into.

SEE ALSO

Sget, Sput, Schksum.