A Software Solution to Optimising Submission and Publication of Oceanographic Data

One of the key objectives of the British Oceanographic Data Centre is to capture, catalogue and preserve a wide range of oceanographic data. It is important to capture metadata alongside the data coming in from originators, allow data managers to curate the (meta)data, and publish it in good time. To address this we have built a new data submission and ingestion application using the latest cutting edge technology. The design of the new system was heavily user focused, capturing feedback on existing processes and addressing outstanding issues faced by users previously. This system makes key improvements to the whole data lifecycle, from metadata capture, data archiving, and Digital Object Identifier (DOI) publication. By creating a more streamlined workflow we have drastically reduced the turnaround time for data publication. The clean, intuitive user interface makes the submission process effortless even for first time users. The application's modular design and use of a non-relational database allows us to handle a range of different data types, including discrete samples and time series, while also having the flexibility to handle new requirements coming from the scientific domain. The technology choices of the architecture were made to prioritise an application that is resilient, reliable, and able to grow with changing requirements. A containerised architecture is employed to simplify deployment, improve scalability, and simplify the development process. RabbitMQ messaging is used for data archiving to move files asynchronously and securely, while giving a seamless user experience.