Loop Closure Detection and SLAM in Vineyards with Deep Semantic Cues

Automation of vineyards cultivation necessitates for mobile robots to retain accurate localization system. The
paper introduces a stereo vision-based Graph-Simultaneous Localization and Mapping (Graph-SLAM) pipeline custom-
tailored to the specificities of vineyard fields. Graph-SLAM is reinforced with a Loop Closure Detection (LCD) based on
semantic segmentation of the vine trees. The Mask R-CNN network is applied to segment the trunk regions of images, on
which unique visual features are extracted. These features are used to populate the bag of visual words (BoVWs) retained
on the formulated graph. A nearest neighbor search is applied to each query trunk-image to associate each unique feature
descriptor with the corresponding node in the graph using a voting procedure. We apply a probabilistic method to select the
most suitable loop closing pair and, upon an LCD appearance, the 3D points of the trunks are employed to estimate the loop
closure constraint to the graph. The traceable features on trunk segments drastically reduce the number of retained BoVWs,
which in turn significantly expedites the loop closure and graph optimization, rendering our method suitable for large scale mapping in vineyards. The pipeline has been evaluated on several data sequences gathered from real vineyards, in different seasons, when the appearance of vine trees vary significantly, and exhibited robust mapping in long distances.