Multimodal Agricultural Aerial and Ground Robotics Simulation Dataset

Dataset description

This dataset was generated using an aerial robot and a ground robot in the Webots simulator with the OpenDR agricultural dataset generator tool.

It consists of 13980 RGB images and their semantic segmentation counterparts taken at different lighting conditions and robot positions in an agricultural field. It also includes the annotation data comprised of the class of the object, x, and y of the top left pixel of the object bounding box, and the width and height of the object bounding box. Furthermore, it includes gps and inertial unit sensor data for UAV and gps, inertial and lidar sensor data for UGV.

Folder configuration

The dataset contains 4 folders for different lighting conditions:

• noon cloudy
• noon stormy
• dawn cloudy
• dusk

Each contains UAV and UGV folders. UAV folder includes:

• annotations: contains segmented images in JPG files and annotations in TXT files.
• camera: contains generated RGB images.
• gps: contains the three-axis location of global positioning sensor saved in TXT files.
• inertial unit: contains the inertial unit date in TXT files.

UGV folder includes:

• annotations: contains segmented images in JPG files and annotations in TXT files.
• front_bottom_camera: contains generated RGB images.
• Hemisphere_v500: contains the three-axis location of the global positioning sensor saved in TXT files.
• imu_robotti: contains the inertial unit date in TXT files.
• velodyne: contains lidar data in PCD files.

Data format

The dataset includes

• The inertial measurement TXT files include Euler angles in order of Roll, Pitch, and Yaw.
• The GPS measurement TXT files include the robot position in x, y, and z order.
• Object annotation TXT files include the class of the object, x, and y of the top left pixel of the object bounding box, and the width and height of the object bounding box at each line for the corresponding frame.

File naming convention

Each data is named "s_i{_segmented, _annotation}.ext", where:

• s denotes the simulated time in seconds.
• i denotes the index counting every 10ms of simulated time.
• ext denotes the extension, "jpg" for images, "pcd" for lidar, and "txt" for the rest.
• Labels _segmented and _annotation appended to the name for segmentation image and object annotations, respectively.

Each segmented image uses the following RGB color mapping:

• Tree: 0.1, 0.4, 0.0
• Apple Tree: 0.85, 0.49, 0.57
• Cow: 0.380, 0.220, 0.137
• Sheep: 0.937, 0.921, 0.862
• Fox: 0.992, 0.376, 0.086
• Barn: 0.625, 0.293, 0.226
• Cat: 0.870, 0.580, 0.0
• Deer: 0.415, 0.364, 0.302
• Human: 1.0, 0.855, 0.672