Pitschi: a Clowder-based end-to-end data management tool

Data management at research facilities that operate a range of big-data-producing instruments has become ever more critical as the amounts of data generated by such instruments keep increasing thanks to technological advances. In this report we present the data management tool Pitschi. Based on the data management framework Clowder, Pitschi has been designed for the Centre of Microscopy and Microanalysis at The University of Queensland as an end-to-end solution that supports the entire research data lifecycle by storing, indexing, and annotating data generated at the centre. The workflow in Pitschi is described, particularly how it is fully integrated with the university’s infrastructure. Importantly, Pitschi can also be adapted to other institutional environments. This work was undertaken under Work Package 4: “Big-data electron and correlative microscopy from instrument to publication” of the Australian Characterisation Commons at Scale (ACCS) project.