When Data Doesn't Fit

Description

Abstract

Modern bioimaging generates data of extraordinary richness — and often extraordinary size. As researchers adopt ever more advanced instruments and techniques, the resulting datasets routinely exceed the capacity of local memory, standard file systems, and traditional analysis workflows. This talk traces the evolution of open-source infrastructure designed to meet this challenge, from early efforts to standardize microscopy metadata (OME-XML, Bio-Formats), to server-based platforms for managing and sharing data (OMERO), to cloud-native formats like OME-Zarr that support scalable storage and computation.

Along the way, we’ll explore the practical obstacles and design decisions that shaped each generation of tooling: not just how to store large image datasets, but how to describe them, share them, and ultimately make them FAIR — Findable, Accessible, Interoperable, and Reusable. The story is not just one of technical progress, but of community-building, evolving standards, and an ongoing effort to ensure that valuable biological data remains usable across disciplines, platforms, and time. As bioimaging converges with other data-intensive fields, the lessons from this journey may resonate more widely — wherever data no longer “fits,” and where infrastructure must rise to meet it.

Notes