UK National Data Library: Distributed Architecture for Research

The UK National Data Library: Technical White Paper Challenge, initiated by Wellcome and the Economic and Social Research Council (ESRC), sought visionary approaches to enhancing access to UK public sector datasets. Evaluated by an expert panel, this paper was recognised as a distinguished and successful response to the call, offering innovative insights for creating a National Data Library that benefits both the research community and the public good.

The data landscape for research and the public sector in the UK today is fragmented into thousands of disconnected databases. Our vision is to establish a National Data Library (NDL) using a distributed architecture that drives growth and improves public services.

Data is plentiful, but the UK lacks data orchestration and governance. The situation is reminiscent of telecommunications before standardised internet protocols, many local implementations which allowed information to flow within local domains, like banks or government but without interoperability. In its simplest form, this paper wants to do for research what TCP-IP did for the internet - create a common data interchange protocol and technical architecture. We make three fundamental decisions: the open standard for how data is stored, the mechanisms for data to be shared, and the governance required to connect trusted parties to these data.

Our technical white paper will outline an operational architecture to ensure ease of access and interoperability while maintaining public trust. By leveraging secure cloud collaboration, template-driven infrastructure architectures, and proven, scalable technologies, we propose a blueprint for an economically viable ecosystem that shares valuable data applications and insights. This paper focuses on user needs, benefits realisation, technical architecture, and sustainable management of the NDL. Together, these can make the UK the global leader in research data infrastructure.