A Dataset of Synthetic Images of Outdoor Scenes Taken from Sidewalks, for Temporal Semantic Segmentation Applications

This dataset has been generated using the CARLA simulator (release 0.9.11), an open-source 3D simulator for experiments in autonomous vehicle, based on the Unreal Engine game engine. It comes with pre-made city environment maps. CARLA is distributed with several integrated maps as well as parameters to increase the variety in the dataset. In the release that we have used, there are 13 semantic segmentation classes: None, Building, Fence, Other, Pedestrian, Pole, Lane-marking, Road, Sidewalk, Vegetation, Vehicle, Wall, and Traffic sign. The "None" category corresponds to textures that are not part of an object, such as lawns which are not part of "Vegetation", or sky. In the “Other” category are found objects that are not included in the other classes like plant and flower pots. For smart mobility applications, the “Sidewalks” and “Road” classes are of particular importance to find the way forward, as well as “Buildings” and “Poles” for obstacle avoidance. Sequences are made of 4 images. The dataset is composed of 46436 frames (11609 sequences) partitioned in 41024 frames (10256 sequences) for train, 2696 frames (674 sequences) for validation, and 2716 for test (679 sequences). The size of the images is 800 x 600 (resp. width x height).

Additionaly, we have generated another smaller dataset with images taken from 2 different viewpoints: one located on the road and the other located on the sidewalk. The number of frames for train/validation/test is respectively 7288 (1822 sequences) partitioned in 6344 (1687 sequences) for train, 416 frames (104 sequences) for validation, and 424 for test (106 sequences). This smaller dataset is aimed at showing the importance of the viewpoint in the result of semantic segmentation. This can be done by cross-validation: learning on images taken from a viewpoint located on the road and test on images with a viewpoint located on the sidewalk, and vice versa.