Anisotropic Diffusion-Based Enhancement of Scene Segmentation with Instance Labels

Many visual scene understanding applications, especially in visual servoing settings, may require high quality object mask predictions for the accurate undertaking of various robotic tasks. In this work we investigate a setting where separate instance labels for all objects under view are required, but the available instance segmentation methods produce object masks inferior to a semantic segmentation algorithm. Motivated by the need to add instance label information to the higher fidelity semantic segmentation output, we propose an anisotropic label diffusion algorithm that propagates instance labels predicted by an instance segmentation algorithm inside the semantic segmentation masks. Our method leverages local topological and color information to propagate the instance labels, and is guaranteed to preserve the semantic segmentation mask. We evaluate our method on a challenging grape bunch detection dataset, and report experimental results that showcase the applicability of our method.