HSI-Drive

Description

Methods

The recording system setup for this project was extremely simple, consisting of just one Photonfocus MV1-D2048x1088-HS02-96-G2 camera. The Photonfocus MV1 camera is a small-size snapshot camera with a GigEVision interface that can run at up to 42fps depending on its configuration. A 12-bit resolution has been used for raw binary information coding, while the camera throughput has been limited to 11fps to avoid excessive memory consumption. The selected optics was an Edmund Optics 16mm C Series VIS-NIR fixed focal length lens. Attached to the MV1, this lens provides a 30.9º FOV.

HSI-Drive contains images and video sequences obtained in diverse scenarios and under diverse environmental conditions. All recordings have been made on roads and in towns in the province of Biscay, in the Basque Country, Spain. The dataset is organized according to four parameters:

1. Season of the year: winter, spring, summer and fall.
2. Day time: dawn, full daylight and sunset.
3. Weather: sunny, cloudy, wet/rainy and foggy.
4. Road type: urban streets and roads, interurban and country roads, and highways.

This dataset is aimed at the development of pixel-level classification systems that directly rely on the separability of the spectral signature of materials and on features obtained from spectral information. The labeling of classes for the image annotation has been performed according to material surface reflectances.

The annotation procedure has been very conservative, manually selecting only the areas that clearly belong to each class, and leaving the edges and some areas of the background unlabeled. This procedure favours that all pixels in a class subset contain only the spectral reflectances of the class concerned. This approach is aimed to maximize ML training based on spectral features to the detriment of techniques that rely on spatial features.

Class definitions for annotation:

• Class1: Road (tarmac)
• Class2: Road marks
• Class3: Vegetation (any kind of vegetation, including wood)
• Class4: Painted metal (road signs, traffic light posts, vehicle bodies etc.)
• Class5: Sky
• Class6: Concrete/stone/brick (infraestructure)
• Class7: Pedestrian/cyclist
• Class8: Water (water courses, puddles etc.)
• Class9: Unpainted metal (back of road signs and signposts, road sign posts, streetlight posts, crash barriers etc.)
• Class10: Glass/transparent plastic (vehicle windscreens, headlights and backlights, windows, traffic lights etc.)

Table of contents

The dataset contains:

• RAW.zip - Original RAW binary files (integer 12 bit).
• Cubes_NoScaling.zip - Spectral cubes (25 bands) with no automaticaly estimated white balancing (float single precission).
• Cubes_NoScaling_MF.zip - Spectral cubes (25 bands) with no automaticaly estimated white balancing and with median filtering for backwards compatibility with v2.0 (float single precission).
• Cubes_Scaling.zip - Spectral cubes (25 bands) with automaticaly estimated white balancing (float single precission).
• refs_WHITE.zip - Reference white images for flat field and reflectance correction . 
• refs_DARK.zip - Reference dark images for bias correction.
• RGB.zip - False RGB images corresponding to each cube for rapid visual inspection.
• Labels.zip - Manually per-pixel annotated ground-truth images for image semantic segmentation.
• HSI_Drive_v2_1_release_notes.pdf - Comments on the new features of the dataset.
• image_list_specs.ods - Metadata about each image in the dataset
• HSI_Drive_21_White_Paper_2025.pdf - Technical details about the data in the HSI-Drive dataset