Published April 23, 2021 | Version 1.0.0
Video/Audio Open

Underwater surveys of mullet schools (Mugil liza) with Adaptive Resolution Imaging Sonar

  • 1. Max Planck Institute of Animal Behaviour
  • 1. Universitat de Barcelona
  • 2. Max Planck Institute of Animal Behaviour

Description

This dataset is part of a research project that employs deep learning, with a density-based regression approach, to count fish in low-resolution sonar images (Tarling et al. preprint arXiv DOI: http://arxiv.org/abs/2104.14964).

In this repository, we provide data from sonar-based underwater videos of schools of migratory mullets (Mugil liza) recorded at the Tesoura beach (28.495775 S, 48.759996 W), a 100-meter long beach at the inlet canal connecting the Laguna lagoon system to the Atlantic Ocean, in southern Brazil. Since the water transparency at the lagoon canal is very low (from 0.3 to 1.5m visibility; collected in situ with a Secchi disk), mullet schools were recorded by deploying an Adaptive Resolution Imaging Sonar, ARIS 3000 (Sound Metrics Corp, WA, USA), which uses 128 beams to project a wedge-shaped volume of acoustic energy and convert their returning echoes into a digital overhead view of the mullet schools.

This dataset contains 500 fully annotated images that were manually marked for the location and abundance of mullet fish, and  126 raw sonar video files, representing over 100k images. The files are organized as follows:

1) "2018-MM-DD_HHMMSS" files are mp4 videos (you may need to add the file extension ".mp4"): There are 126 ARIS files converted into MP4 videos totalling over 789MB of underwater footage captured at 3 frames/seconds. Note that file names indicate the date and time the video was recorded.

2) ".npy" files (in Labelled_data.zip): From the video files, 500 images were selected for labelling. Images (x) were cropped to represent a 4x8.5m2 area and resized to 320 x 576 pixels. Mullet fish were marked with a point annotation. Corresponding ground truth density maps (y) were generated by convolving a Gaussian kernel over the image mask, size =4 and standard deviation = 1. The labelled dataset was randomly split into a holdout partition of 350 training images, 70 validation, and 80 test. 

3) ".csv" files: log of frames selected for the labelled subset of data

4) ".h5" file: pre-trained weights for our multi-task with uncertainty regularisation network

To advance the development of these machine learning tools, we also make our code openly available (https://github.com/ptarling/DeepLearningFishCounting).

Notes

The data sampling was supported by research grants from the National Geographic Society (Discovery Grant WW210R-17) and post-doctoral fellowships from Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES Brazil; #88881.170254/2018-01) and Conselho Nacional de Pesquisa e Desenvolvimento Tecnológico (CNPq Brazil; #153797/2016-9). The research on the machine learning tools has been partially supported by the Spanish project PID2019-105093GB-I00 (MINECO/FEDER, UE) and CERCA Programme/Generalitat de Catalunya, and by ICREA under the ICREA Academia programme awarded to Sergio Escalera. Mauricio Cantor is currently supported by The Max Planck Society via the Department for the Ecology of Animal Societies at the Max Planck Institute of Animal Behaviour, and grants from the CAPES-DAAD PROBRAL Research Programme (#23038.002643/2018-01) and the SELA CNPq-PELD Research Program (SELA 445301/2020-1). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the dataset.

Files

Labelled_data.zip

Files (2.3 GB)

Name Size Download all
md5:de6dbaa1c24942381b5294545d1af4e0
1.3 MB Download
md5:fe68ad504b91b0eb47ed38435069387e
1.6 MB Download
md5:b557036ac85e6e96da450b51a3987034
638.1 kB Download
md5:435b608c10e1d7e13026593d660e7d95
477.0 kB Download
md5:61f0be79f20af53db5f2b85e4f2c8b7e
4.6 MB Download
md5:72381d6281736512872b17629575667a
7.3 MB Download
md5:aa4609d26a336275fc6f5b53a7acde66
140.3 kB Download
md5:366224f0984d7d008ad33c08b8ec751f
3.9 MB Download
md5:624fa903c3739922da00c7635eff618c
4.2 MB Download
md5:d9af7d461350d54a9d1007d92c78b573
281.3 kB Download
md5:374c10bffe8c6ef62a6cb690e351ecf4
7.3 MB Download
md5:1f1c1571255c53eed852bb0c4565ed7d
287.1 kB Download
md5:69ab1175d81b50654a8b611e29b4f44a
5.1 MB Download
md5:beb058882f98416eea6f6b5a74b4d682
12.5 MB Download
md5:5de0d61afa4af04c070eaf8c831bb748
1.9 MB Download
md5:a57e243e329da7a1caf31e75f59c3ebc
2.2 MB Download
md5:fad8c56e073d674edf37f176593f22a7
17.6 MB Download
md5:b4f61ba30f45a1d35bad696b53315663
121.8 kB Download
md5:4cf26855eaece985177a97f75f555d3f
4.9 MB Download
md5:58504c1223052d60679756eb1fcaf336
4.3 MB Download
md5:42de93ad3732b03e407b43e50b76ec34
8.6 MB Download
md5:a26997e9aa85663f52460c07bacc7adc
11.5 MB Download
md5:054251b035fa8cea5adb8d344c72ac79
937.1 kB Download
md5:0dd6fe828f5e4b82007804f76f73c3bb
100.1 kB Download
md5:78b2a5a72da78ebb67f65d67f9083036
4.6 MB Download
md5:239d7de6d45a1d384e9daaeae2231240
555.2 kB Download
md5:108f5c64e5e3faeb3763a73169a7cd40
2.8 MB Download
md5:ea820882f07f677f98e792908c9db6ee
2.8 MB Download
md5:25ac2398802c6235760887acc0432e79
3.7 MB Download
md5:b84cab7d6b3d9fdc0499e3571609450b
6.4 MB Download
md5:b5730a6ba3578d76ab56a0aa92c155da
3.2 MB Download
md5:d16ec9d39e99645bbf8c824ef59bb5b9
20.4 kB Download
md5:19edf1bb68c5fec401d6b6c3407e034b
2.6 MB Download
md5:2f7efff70bc46173ce5636be32eead9c
1.1 MB Download
md5:38fbc1a594981ca137d8981570febf4a
403.2 kB Download
md5:b9c6f085fe3d1d0678d4e37d05e5122a
2.0 MB Download
md5:0ddf6755d28afa85c53eeea6f781d6f1
1.5 MB Download
md5:567b57fe2ed51047e326feed40a903f9
15.0 MB Download
md5:001a7716d192a7b5334118a464070a15
25.1 kB Download
md5:7f252644d85375b8c8ae8485d6350178
8.8 MB Download
md5:c03014ce1f01a6e7e5718b2cca8947e8
2.2 MB Download
md5:c4403eab33603527c95b4641b5588fcc
2.9 MB Download
md5:0d27c1b6a8f0a5bb710502e81d5fd3e5
5.3 MB Download
md5:7686102ccb8a7773ef968a35c033974b
9.3 MB Download
md5:c7fc60a5131463741931a1db8d471d88
2.5 MB Download
md5:f0ebd6f47e486aa1b0823a22b5a8b115
4.7 MB Download
md5:b732f9a8baa69d6cffd895a83dde22d0
2.2 MB Download
md5:a9987e147c421a1f1ccbbebdc3500e0d
1.5 MB Download
md5:95e1cb9f6806de2b46ac7679059732c2
2.6 MB Download
md5:821325cfd19d8925af6b461f939863e3
1.7 MB Download
md5:e5f3a7df3abe2c0545940942eeef7653
9.4 MB Download
md5:77bb0de77e3cf638e5b32b7c08b52ea7
30.8 kB Download
md5:5147ef361e409a29a7ad0ae79804ae8d
404.4 kB Download
md5:247389805ef8a28660c5d820abc69143
11.6 MB Download
md5:d2a2f402f532b5cff0834deb31e902cf
3.1 MB Download
md5:c94e296a8bc714a4daff7abdf1f682ef
651.7 kB Download
md5:e8be7068508a5cae9bbc2f838a8a34d1
5.6 MB Download
md5:e305f935b96f293e751e4d0c039f74d9
6.6 MB Download
md5:7e36caf0659f7e93dbbce86e7442b45a
8.4 MB Download
md5:79f3b5a35be7e49e4d269961dc661960
13.0 MB Download
md5:ff8d9dc3dd489f3466498afed29142d6
7.1 MB Download
md5:3e0ca0739d1677a02aad6fdfe3782cbc
2.7 MB Download
md5:86b5142f326e6b2a39c9e963c185aa79
4.7 MB Download
md5:582478001e56d1df005674adab7972e0
34.9 kB Download
md5:aed067c052d30e5b9351e5f3ffb75549
15.6 kB Download
md5:c22cb10154b4a357b764ca339e760ec5
4.4 MB Download
md5:8b1aa6c7ce0e7ace7adef43d696db1c4
3.4 MB Download
md5:36018238087648e9189e054d1a7b70bd
6.0 MB Download
md5:ee825c9162a054b9f835a70937236dcb
2.9 MB Download
md5:bb18ca9277ea9a052c9768f942a087ad
1.3 MB Download
md5:390615de9da32455857ab4b1c291df79
3.1 MB Download
md5:730b690e03ea4813cf94def8335563ff
3.8 MB Download
md5:48ca83acccb3fca4b57c5d0d2544a207
3.6 MB Download
md5:430e22f727b0b1d886ead8c8f0057e8a
2.3 MB Download
md5:7f1bcda485b5c74fdbb6cf562cc91ece
9.1 MB Download
md5:3815ba7a06abb99d0550ac41a059ae0b
4.7 MB Download
md5:c3bd13dd96c044578742d72a1aa4b847
4.3 MB Download
md5:be56dbdc92d423410ed72950263f9b88
14.5 MB Download
md5:d7d27ee46e19f120a921dbe03bca6a6d
4.4 MB Download
md5:04cd2278c101d2037e56a9e6d6032e90
399.9 kB Download
md5:dc4da8de605c0be2db0bc242af2c6ef3
2.7 MB Download
md5:fd7d61f33c67ac980f14586c7e8e91a1
4.0 MB Download
md5:aebca115905959a0b9d96e3439ef1df5
4.2 MB Download
md5:8b594cdf023cbb2459b4bda9d9b14656
3.0 MB Download
md5:527e81fba8ac8d61f532ab6f49273f54
5.1 MB Download
md5:0ef70505884caf56ae2ef358ae98b214
2.7 MB Download
md5:4c22fb4614219a6574fe4a249b5faed1
4.2 MB Download
md5:86e652de5eacefab44dd4e8940806d18
3.8 MB Download
md5:ba4f71714c1527e448bfd18e35169ce9
5.1 MB Download
md5:5c1db0ba982d850114913d020060249b
3.9 MB Download
md5:363361e4c966f0bec1dac77bec3c0738
6.1 MB Download
md5:455e13cc88ea689f6db4b89c02dc22bc
12.1 MB Download
md5:334d2f3801b145012c6d1d68c58b5645
7.4 MB Download
md5:45b2c4a1951087ca550ee8ed4192ae62
5.2 MB Download
md5:72b98c129d63cb3d7b6cb7e66d72bff2
3.3 MB Download
md5:d22bf5eff2fab7d07dd9e47271d10760
3.9 MB Download
md5:ba335881b2160f920edf4ced8a999666
9.4 MB Download
md5:0e4a36c28edf29292e53cb45fa5132ef
6.1 MB Download
md5:10f528894d7f24b75a9fc8081555159f
3.9 MB Download
md5:1178f62fc68c23f7a1597161818905b2
13.1 MB Download
md5:a52c32e28b6a8edb344622f114b11d0f
848.4 kB Download
md5:b6f44c8cc3b41830a7626d8ca9063095
4.2 MB Download
md5:1ae79449d4add17796cc3ea4f6618a72
4.6 MB Download
md5:68fb698e7a73791df666e4de85b6e67d
6.6 MB Download
md5:07c53f3326d260d26d6ff1868f159095
6.9 MB Download
md5:838a02073336efcffcd0b594f7a6dcd3
5.3 MB Download
md5:76117af01c961ed5ed11ac835ae74491
4.1 MB Download
md5:a19e7cdfcab1fc3222a9f94d9a78173e
5.1 MB Download
md5:5ccb6aea3587b79181fe50bbeea58c1a
8.7 MB Download
md5:7876d03e6494d0888002e37b636ba338
12.3 MB Download
md5:f741c851e328099811db119a94cbaaef
8.1 MB Download
md5:43dc09f23379344edc26864606a7e687
8.5 MB Download
md5:3758340da225b5d8ce2fbc4e967774c3
12.9 MB Download
md5:a996a48c137e5db26a242689442c7e66
8.6 MB Download
md5:622004140ef30302316022db6554403f
7.9 MB Download
md5:67540428ffa4e7f08d1d33600b6904ad
7.7 MB Download
md5:b5ba67058f04b93cc7c1e71e776a14a5
8.8 MB Download
md5:d2d10778f9eb2cf5d282966bc3fe5d59
10.2 MB Download
md5:b8d348302f624fe5d8894c8ae861933f
7.6 MB Download
md5:06a3386136c82adb638a56128d850f3a
24.2 MB Download
md5:75230e2651ab84e440e063fda08fc0ce
9.2 MB Download
md5:09b6564af0e50dbbe3c402473a0aa27b
11.2 MB Download
md5:76703beba69a67014c803dea470e56bc
7.7 MB Download
md5:bbcdeabd303c595b5b125f4cd7eefb89
10.2 MB Download
md5:6a2bfdef0d13ca13994e9f4c7143db8d
9.2 MB Download
md5:d401a1f1a211b0d5b9d10485505bbbfe
3.1 MB Download
md5:0bf7bb5d34093ff155b7e1b16fb3b2ac
6.6 MB Download
md5:3923ccae4005db7192640a2179e6fb59
28.9 MB Preview Download
md5:26be3670b95102a9dcdd96eecfde0c10
3.5 kB Preview Download
md5:79aced1e19755b68bc828537fdfc950b
2.5 kB Preview Download
md5:a15f35a50a8f16965eb172f112b28632
11.0 kB Preview Download
md5:b9ea348e2e78b169fc4d4454ab519e0b
2.2 kB Preview Download
md5:2aa436ab54dd6ad199b7a3cbb9bc7d55
94.6 MB Download
md5:33b58fd0c4391b8913aef914a315d5b7
1.5 GB Download

Additional details

References

  • Tarling, P., Cantor, M., Clapés, A., & Escalera, S. (2022). Deep learning with self-supervision and uncertainty regularization to count fish in underwater images. PloS one, 17(5), e0267759. doi: https://doi.org/10.1371/journal.pone.0267759
  • Tarling P, Cantor M, Clapés A, Escalera SG. Deep learning with self-supervision and uncertainty regularization to count fish in underwater images. arXiv: http://arxiv.org/abs/2104.14964
  • Tarling, P. (2021). Deep learning to count fish in sonar images. MSc dissertation. Universitat de Barcelona. Permanent URL: http://hdl.handle.net/2445/186155
  • Cantor, M., Farine, D. R., & Daura-Jorge, F. G. (2023). Foraging synchrony drives resilience in human–dolphin mutualism. Proceedings of the National Academy of Sciences, 120(6), e2207739120.