Fusion of Underwater Camera and Multibeam Sonar for Diver Detection and Tracking

 

This dataset is related to previously published public dataset "Sonar-to-RGB Image Translation for Diver Monitoring in Poor Visibility Environments". https://zenodo.org/records/7728089

It contains ZED-right camera and sonar images collected from Hemmoor Lake and DFKI Maritime Exploration Hall.

 

Sensors: Low Frq (1.2MHz) Blueprint Oculus M1200d Sonar and ZED Right Camera

 

 

The dataset is created for Diver Detection and Diver Tracking applications.

 

For the Diver Detection part, the dataset is prepared to train, validate and test YOLOv7 model.

7095 images are used for training data, and 3095 images are used for validation data. These sets are augmented from originally captured and sampled ZED camera images. Augmentation methods are not applied to the Test data, which contains 822 images. Train and validation contain images from both the DFKI pool and Hemmor Lake, while the test data is only collected from the lake.

 

To distinguish between the original image and the augmented image, check the name coding. 

Naming of object detection images:

original_image_name.jpg

if augmented:

original_image_name_<augmentation_number_of_the_same_image>.jpg

 

Object Detection Label Format: 

YOLO [(class), ((x_min + (x_max - x_min)/2)  / image_width), ((y_min + (y_max - y_min)/2)  / image_height), ((x_max - x_min) / image_width), ((y_max - y_min) / image_height)]

 

Class: "diver", represented by "0" in object detection labels.

 

Resolution of Object Detection Camera Images: 640x640

Resolution of Object Tracking Camera Images: 1280x720

Resolution of Object Tracking Low Frequency Sonar: 932x514

 

About the Object Tracking on Sonar, the sampled data is the part where diver moves around the table and the platform. 

There are 4 cases shared in the dataset, which contain a sonar stream, and corresponding ZED-right camera images. 

Totally, 1193 points represent the diver on sonar images for the diver tracking application.

 

For the tracking, "tracking_sonar_coordinates_<number>.csv" contains x,y coordinates of a point where the diver is in the sonar image. 

And "image_sonar_<number>.csv" file contains the matching between sonar and camera images.

 

 

The data in this repository were collected as a joint effort between the German Center for Artificial Intelligence (DFKI), the German Federal Agency for technical Relief (THW), and Kraken Robotics GmbH. This work is part of the project DeeperSense that received funding from the European Commission. Program H2020-ICT-2020-2 ICT-47-2020 Project Number: 101016958.