Published May 3, 2021 | Version v1
Book chapter Open

Sensor Data Fusion and Autonomous Unmanned Vehicles for the Protection of Critical Infrastructures

  • 1. Information Technologies Institute-Centre for Research and Technology Hellas, Thessaloniki, Greece


Current technology in imaging sensors offers a wide variety of information that can be extracted from
an observed scene. Acquired images from different sensor modalities exhibit diverse characteristics such as type
of degradation; salient features etc. and can be particularly beneficial in surveillance systems. Such representative
sensory systems include infrared and thermal imaging cameras, which can operate beyond the visual spectrum
providing functionality under any environmental conditions. Multi-sensor information is jointly combined to pro-
vide an enhanced representation, particularly utile in automated surveillance systems such as monitoring robotics.
In this chapter, a surveillance framework based on a fusion model is presented in order to enhance the capabilities
of unmanned vehicles for monitoring critical infrastructures. The fusion scheme multiplexes the acquired repre-
sentations from different modalities by applying an image decomposition algorithm and combining the resulted
sub-signals via metric optimization. Subsequently, the fused representations are fed into an identification module
in order to recognize the detected instances and improve eventually the surveillance of the required area. The
proposed framework adopts recent advancements in object detection for optimal identification by deploying a deep
learning model properly trained with fused data. Initial results indicate that the overall scheme can accurately
identify the objects of interest by processing the enhanced representations of the fusion scheme. Considering that
the overall processing time and the resource requirements are kept in low levels, the framework can be integrated
in an automated surveillance system comprised by unmanned vehicles.


Fusion for UVs_KIoannidis.pdf

Files (742.5 kB)

Name Size Download all
742.5 kB Preview Download

Additional details


beAWARE – Enhancing decision support and management services in extreme weather climate events 700475
European Commission
ROBORDER – autonomous swarm of heterogeneous RObots for BORDER surveillance 740593
European Commission