Skip to content

Archive Migration

(HPSS to SAM-QFS 2009)

July 6, 2009; SDSC is planning to consolidate from two archival storage systems to one over the coming months. After careful consideration, we have chosen SAM-QFS as our ongoing archive storage solution and will be phasing out HPSS. SAM-QFS provides a complete data management system consisting of two integrated software products: SAM (Storage Archive Manager) and QFS (a high-performance, 64-bit, Solaris file system). Using SAM-QFS, users may access data directly using a disk cache filesystem; the data then automatically migrates to tape. In approximately a month, HPSS will transition to read-only status and will remain read-only until the migration to SAM-QFS is complete.

SDSC staff will be responsible for moving all the data from HPSS to SAM-QFS and will provide documentation and assistance for HPSS users to access their data in SAM-QFS post-migration. Users are requested NOT to move their own data, as this will significantly delay the migration effort. SDSC TeraGrid staff will be directly contacting allocated TeraGrid users with significant HPSS archival storage to request confirmation of data to be migrated to SAM-QFS. (HPSS data for smaller users will automatically be migrated, although we also encourage these users to review their holdings.) In addition, this consolidation is an important opportunity to clean up data that are no longer necessary.

SDSC intends to retain existing TeraGrid user data in SAM-QFS for at least four years (June 2013) to facilitate a smooth transition from TeraGrid to the follow-on XD program and to any subsequent programs for robust longer-term archival storage. Existing storage allocations will be honored and new requests for simulation data from SDSC HPC systems will be supported. However, storage of new archival data by TeraGrid HPC users will require an explicit storage request through the TeraGrid’s allocation process. The TeraGrid Resource Catalog will describe acceptable uses of SDSC Tape Storage; contact help at teragrid dot org with any questions.

HPSS archival data for non-TeraGrid users will also be migrated to SAM-QFS, and new data can be written into SAM-QFS. However, a recharge will be implemented for archival storage by non-TeraGrid users over the coming months, and we strongly urge users to prune their holdings prior to migration and retain only necessary data in anticipation of the recharge. Further information about costs and schedule for this recharge will be provided to non-TeraGrid users as the information becomes available.

For concerns or questions please contact SDSC consulting staff at 1-866-336-2357 or consult@sdsc.edu.


Did You Get
What You
Wanted?
Yes No
Comments