Semantic scene understanding and traversability estimation for off-road vehicles

In recent years a lot of research has been carried out by big tech companies in the field of autonomous driving. However, this scientific exploration has been conducted specially in cities or suburban areas and less attention has been given to unstructured off-road situations.

To operate autonomously and intelligently in unstructured terrain, a robot or vehicle must possess many advanced navigational skills. In particular, it needs to have capability to perceive its surrounding environment, recognize hazardous terrain topography, discriminate negotiable and non-negotiable obstacles, and select suitable routes compatible with its mobility capabilities without jeopardizing its safety and mission. Therefore, the goal of this project is to develop a spatial AI algorithm for off-road vehicles that is able to perceive and understand autonomously the real physical world by identifying the surrounding main surface and object classes (grass, meadow, dirt road, trail, bush, tree, etc.) and estimate their traversability using RGB and/or multispectral camera and/or LiDAR sensors.

Self driving vehicles require a deep understanding of their surroundings to a high-level pixelwise accuracy. Hence, the aforementioned AI algorithm will probably consist of a semantic segmentation model, which assigns a predefined class to each pixel of an image. These models are usually built using Deep Learning techniques and, in particular, convolutional networks have been demonstrated to work notably well with image related problems. A further literature review will be conducted along with several tests and evaluation in order to identify an optimal architecture for this model to perform accurately in real-time scenarios.