Data Everywhere: Using and Sharing Scientific Data with Pelican

While there are perhaps hundreds of petabytes of datasets available to researchers, instead of swimming in seas of data there is often a feel of sitting in a data desert: there’s a mismatch between what sits in carefully curated repositories around the world versus what’s accessible at the computational resources locally available. The Pelican Project (https://pelicanplatform.org/) aims to bridge the gap between repositories and compute by providing a software platform to connect the two sides. Pelican’s flagship instance, the Open Science Data Federation (OSDF), serves billions of objects and more than a hundred petabytes a year to national-scale resources. This tutorial, targeted at end-user data consumers and data providers, covers the data access model of Pelican, guides participants to access and share data through an existing data federation, and considers how data movement via Pelican and the OSDF can enable their research computing.