A Framework for Analysis of Aquatic Ecosystems

State-of-the-art underwater imaging systems provide an exciting opportunity to observe billions of individual organisms in their natural habitats at unprecedented spatiotemporal resolution. To unlock the full potential of these advances, we require new analysis pipelines that go beyond classifying organisms by taxonomic groups, and quantify functional traits and biological phenomena from images. Critically, these tools must be made accessible to domain specialists without programming expertise and deployable at scale on modern supercomputing systems. We develop such an image analysis pipeline, manually annotate functional groups, traits and biological processes in images, and train convolutional neural networks (CNNs) to automate and scale analysis of massive zooplankton image datasets. Our pipeline, implemented on a high-performance computing (HPC) system and combining multiple existing open-source frameworks and libraries, provides an intuitive web interface for browsing, searching and annotating images, and allows multiple simultaneous users to work on a single copy of the data online. Images and annotations are then used for both supervised and unsupervised training of convolutional neural networks (CNNs), with the results made available in the web interface. We demonstrate this approach by classifying ~700,000 images to identify functional groups (copepods, diatom chains, Noctiluca scintillans, marine snow, etc). Organisms are further annotated for relevant functional traits. Using these trait annotations, future work will further train CNNs for object detection and feature extraction, thereby iteratively fine-tuning CNNs to perform increasingly complex trait extraction from images. We foresee that these tools will enable new avenues of investigation in aquatic research, ecosystem modelling and global biogeochemical flux estimations, revealing previously inaccessible relationships between species biodiversity, zooplankton traits and seasonal variations in environmental conditions.