MSL Curiosity Rover Images with Science and Engineering Classes

Data Set Description

The data set consists of 6,820 images that were collected by the Mars Science Laboratory (MSL) Curiosity Rover by three instruments: (1) the Mast Camera (Mastcam) Left Eye; (2) the Mast Camera Right Eye; (3)  the Mars Hand Lens Imager (MAHLI). With the help from Dr. Raymond Francis, a member of the MSL operations team, we identified 19 classes with science and engineering interests (see the "Classes" section for more information), and each image is assigned with 1 class label. We split the data set into training, validation, and test sets in order to train and evaluate machine learning algorithms. The training set contains 5,920 images (including augmented images; see the "Image Augmentation" section for more information); the validation set contains 300 images; the test set contains 600 images. The training set images were randomly sampled from sol (Martian day) range 1 - 948; validation set images were randomly sampled from sol range 949 - 1920; test set images were randomly sampled from sol range 1921 - 2224. All images are resized to 227 x 227 pixels without preserving the original height/width aspect ratio.

Directory Contents

• images - contains all 6,820 images
• class_map.csv - string-integer class mappings
• train-set.txt - label file for the training set
• val-set.txt - label file for the validation set
• test-set.txt - label file for the test set

The label files are formatted as below:

"Image-file-name class_in_integer_representation"

Labeling Process

Each image was labeled with help from three different volunteers (see Contributor list). The final labels are determined using the following processes:

• If all three labels agree with each other, then use the label as the final label.
• If the three labels do not agree with each other, then we manually review the labels and decide the final label.
• We also performed error analysis to correct labels as a post-processing step in order to remove noisy/incorrect labels in the data set. 

Classes

There are 19 classes identified in this data set. In order to simplify our training and evaluation algorithms, we mapped the class names from string to integer representations. The names of classes, string-integer mappings, distributions are shown below:

Class name, counts (training set), counts (validation set), counts (test set), integer representation

Arm cover, 10, 1, 4, 0

Other rover part, 188, 11, 10, 1

Artifact, 664, 60, 132, 2

Nearby surface, 1524, 72, 187, 3

Close-up rock, 1456, 52, 84, 4

DRT, 8, 4, 6, 5

DRT spot, 196, 0, 7, 6

Distant landscape, 348, 14, 34, 7

Drill hole, 252, 5, 12, 8

Night sky, 40, 5, 4, 9

Float, 154, 5, 1, 10

Layers, 178, 21, 17, 11

Light-toned veins, 48, 4, 27, 12

Mastcam cal target, 124, 12, 29, 13

Sand, 234, 19, 16, 14

Sun, 190, 5, 19, 15

Wheel, 212, 5, 5, 16

Wheel joint, 62, 1, 5, 17

Wheel tracks, 32, 4, 1, 18

Image Augmentation

Only the training set contains augmented images. 3,920 of the 5,920 images in the training set are augmented versions of the remaining 2000 original training images. Images taken by different instruments were augmented differently. As shown below, we employed 5 different methods to augment images. Images taken by the Mastcam left and right eye cameras were augmented using a horizontal flipping method, and images taken by the MAHLI camera were augmented using all 5 methods. Note that one can filter based on the file names listed in the train-set.txt file to obtain a set of non-augmented images.

• 90 degrees clockwise rotation (file name ends with -r90.jpg)
• 180 degrees clockwise rotation (file name ends with -r180.jpg)
• 270 degrees clockwise rotation (file name ends with -r270.jpg)
• Horizontal flip (file name ends with -fh.jpg)
• Vertical flip (file name ends with -fv.jpg)

Acknowledgment

The authors would like to thank the volunteers (as in the Contributor list) who provided annotations for this data set. We would also like to thank the PDS Imaging Note for the continuous support of this work.