Sensor Data Fusion and Autonomous Unmanned Vehicles for the Protection of Critical Infrastructures

Current technology in imaging sensors offers a wide variety of information that can be extracted from
an observed scene. Acquired images from different sensor modalities exhibit diverse characteristics such as type
of degradation; salient features etc. and can be particularly beneficial in surveillance systems. Such representative
sensory systems include infrared and thermal imaging cameras, which can operate beyond the visual spectrum
providing functionality under any environmental conditions. Multi-sensor information is jointly combined to pro-
vide an enhanced representation, particularly utile in automated surveillance systems such as monitoring robotics.
In this chapter, a surveillance framework based on a fusion model is presented in order to enhance the capabilities
of unmanned vehicles for monitoring critical infrastructures. The fusion scheme multiplexes the acquired repre-
sentations from different modalities by applying an image decomposition algorithm and combining the resulted
sub-signals via metric optimization. Subsequently, the fused representations are fed into an identification module
in order to recognize the detected instances and improve eventually the surveillance of the required area. The
proposed framework adopts recent advancements in object detection for optimal identification by deploying a deep
learning model properly trained with fused data. Initial results indicate that the overall scheme can accurately
identify the objects of interest by processing the enhanced representations of the fusion scheme. Considering that
the overall processing time and the resource requirements are kept in low levels, the framework can be integrated
in an automated surveillance system comprised by unmanned vehicles.