A decoupled Custodial Copy for cloud-based Digital Preservation Systems

Many archives are making use of cloud-based managed services for Digital Preservation and this can provide a pragmatic way of delivering the long-term storage and active management of digital records without reliance on in-house technical expertise and infrastructure. But not all organisations are comfortable with fully outsourcing the responsibility for holding digital records, and may also worry about topics such as exit planning when moving to a new supplier or business continuity and disaster recovery in case of short or long-term issues with the cloud platform or provider. Generally, some archives will be more comfortable if they also hold their own copy of their digital records in a form that can be read without need for proprietary systems or products.

The National Archives (UK) is making use of a cloud-based managed service for the actively managed and immediately accessible copy of their records but has also built a Custodial Copy system which automatically and immediately takes a copy of all records as they appear in the cloud-based system, and sends a copy to decoupled off-cloud storage. These digital records won’t just be an IT backup of the cloud platform but will be stored in an product-agnostic standardised form, using the Oxford Common File Layout, and with each file accompanied by an up-to-date copy of all the metadata describing the record. This Custodial Copy is intended to be a fully complete and self-describing representation of the digital archive which could be used as a source for future migrations or for access to these records outside of the cloud platform. This poster describes some of the challenges which have been encountered when designing and implementing this capability, and the solutions that have been developed to address these. All the developed software is published on GitHub as Open Source and it is hoped that other archives will benefit from this work.