10.5281/zenodo.4280739
https://zenodo.org/records/4280739
oai:zenodo.org:4280739
Wei Yao
Wei Yao
German Aerospace Center (DLR)
Anastasia Moumtzidou
Anastasia Moumtzidou
Information Technologies Institute, Centre for Research and Technology Hellas (CERTH)
Corneliu Octavian Dumitru
Corneliu Octavian Dumitru
German Aerospace Center (DLR)
Stelios Andreadis
Stelios Andreadis
Information Technologies Institute, Centre for Research and Technology Hellas (CERTH)
Ilias Gialampoukidis
Ilias Gialampoukidis
Information Technologies Institute, Centre for Research and Technology Hellas (CERTH)
Stefanos Vrochidis
Stefanos Vrochidis
Information Technologies Institute, Centre for Research and Technology Hellas (CERTH)
Mihai Datcu
Mihai Datcu
German Aerospace Center (DLR)
Ioannis Kompatsiaris
Ioannis Kompatsiaris
Information Technologies Institute, Centre for Research and Technology Hellas (CERTH)
Early and Late Fusion of Multiple Modalities in Sentinel Image Retrieval
Zenodo
2020
2020-11-19
eng
10.5281/zenodo.4280738
https://zenodo.org/communities/eu
accepted
Creative Commons Attribution 4.0 International
Discovering potential concepts and events by analyzing Earth Observation (EO) data may be supported by fusing other distributed data sources such as non-EO data, for instance, in-situ citizen observations from social media. The retrieval of relevant information based on a target query or event is critical for operational purposes, for example, to monitor flood events in urban areas, and crop monitoring for food security scenarios. To that end, we propose an early fusion (low-level features) and late fusion (high-level concepts) mechanism that combines the results of two EU-funded projects for information retrieval in Sentinel imagery and social media data sources. In the early fusion part, the model is based on active learning that effectively merges Sentinel-1 and Sentinel-2 bands, and assists users to extract patterns. On the other hand, the late fusion mechanism exploits the context of other geo-referenced data such as social media retrieval, to further enrich the list of retrieved Sentinel image patches. Quantitative and qualitative results show the effectiveness of our proposed approach.
European Commission
10.13039/501100000780
776019
EOPEN: opEn interOperable Platform for unified access and analysis of Earth observatioN data
European Commission
10.13039/501100000780
776193
Copernicus Access Platform Intermediate Layers Small Scale Demonstrator