Info: Zenodo’s user support line is staffed on regular business days between Dec 23 and Jan 5. Response times may be slightly longer than normal.

Published September 4, 2024 | Version v1
Presentation Open

Migrating data without original checksums

  • 1. Nasjonalbiblioteket

Description

The National Library (NLN) has been using SAM-FS (Oracle HSM) as a bit-repository since 2007. SAM-FS is soon reaching "EOL". The NLN has recently developed a front-end software solution called DPS (Digital Preservation Services). DPS uses HPSS from IBM as underlying bit-repository. DPS/HPSS is intended to replace SAM-FS as the preservation solution for digital objects. DPS requires that all objects must be delivered with associated checksums. 

In the old SAM-FS bit-repository, many objects lack checksums, especially material from the first years of its use. All objects in SAM-FS are stored in 3 instances. If differences were to be uncovered between the 3 instances, there are no checksums to verify which instance is correct. The total amount of data to be migrated from SAM-FS to DPS is approximately 14 Petabytes. It is estimated that about 1/3 of these data lacks checksums. 

Challenge: How could we ensure that objects migrated from SAM-FS to DPS are the same as those originally archived in SAM-FS when original checksums do not exist? 

Files

Files (1.9 MB)

Name Size Download all
md5:0329bbf56a3f628818608ae87ce0d5df
1.9 MB Download

Additional details

Dates

Submitted
2024-09-19