Object detection and augmented reality annotations for increased situational awareness in light smoke conditions

Innovative technologies powered by Computer Vision algorithms can aid first responders, increasing their situ ational awareness. However, adverse conditions, such as smoke, can reduce the efficacy of such algorithms by degrading the input images. This paper presents a pipeline of image de-smoking, object detection, and augmented reality display that aims to enhance situational awareness in smoky conditions. A novel smoke-reducing deep learning algorithm is applied as a preprocessing step, before state-of-the-art object detection. The detected objects and persons are highlighted in the user’s augmented reality display. The proposed method is shown to increase detection accuracy and confidence. Testing in realistic environments provides an initial evaluation of the method, both in terms of image processing and of usefulness to first responders.