Underwater surveys of mullet schools (Mugil liza) with Adaptive Resolution Imaging Sonar

This dataset is part of a research project that employs deep learning, with a density-based regression approach, to count fish in low-resolution sonar images (Tarling et al. preprint arXiv DOI: http://arxiv.org/abs/2104.14964).

In this repository, we provide data from sonar-based underwater videos of schools of migratory mullets (Mugil liza) recorded at the Tesoura beach (28.495775 S, 48.759996 W), a 100-meter long beach at the inlet canal connecting the Laguna lagoon system to the Atlantic Ocean, in southern Brazil. Since the water transparency at the lagoon canal is very low (from 0.3 to 1.5m visibility; collected in situ with a Secchi disk), mullet schools were recorded by deploying an Adaptive Resolution Imaging Sonar, ARIS 3000 (Sound Metrics Corp, WA, USA), which uses 128 beams to project a wedge-shaped volume of acoustic energy and convert their returning echoes into a digital overhead view of the mullet schools.

This dataset contains 500 fully annotated images that were manually marked for the location and abundance of mullet fish, and  126 raw sonar video files, representing over 100k images. The files are organized as follows:

1) "2018-MM-DD_HHMMSS" files are mp4 videos (you may need to add the file extension ".mp4"): There are 126 ARIS files converted into MP4 videos totalling over 789MB of underwater footage captured at 3 frames/seconds. Note that file names indicate the date and time the video was recorded.

2) ".npy" files (in Labelled_data.zip): From the video files, 500 images were selected for labelling. Images (x) were cropped to represent a 4x8.5m² area and resized to 320 x 576 pixels. Mullet fish were marked with a point annotation. Corresponding ground truth density maps (y) were generated by convolving a Gaussian kernel over the image mask, size =4 and standard deviation = 1. The labelled dataset was randomly split into a holdout partition of 350 training images, 70 validation, and 80 test. 

3) ".csv" files: log of frames selected for the labelled subset of data

4) ".h5" file: pre-trained weights for our multi-task with uncertainty regularisation network

To advance the development of these machine learning tools, we also make our code openly available (https://github.com/ptarling/DeepLearningFishCounting).