Conference paper Open Access

A hybrid graph-based and non-linear late fusion approach for multimedia retrieval

Gialampoukidis Ilias; Moumtzidou Anastasia; Liparas Dimitris; Vrochidis Stefanos; Kompatsiaris Ioannis

Nowadays, multimedia retrieval has become a task of high importance, due to the need for efficient and fast access to very large and heterogeneous multimedia collections. An interesting challenge within the aforementioned task is the efficient combination of different modalities in a multimedia object and especially the fusion between textual and visual information. The fusion of multiple modalities for retrieval in an unsupervised way has been mostly based on early, weighted linear, graph-based and diffusion-based techniques. In contrast, we present a strategy for fusing textual and visual modalities, through the combination of a non-linear fusion model and a graph-based late fusion approach. The fusion strategy is based on the construction of a uniform multimodal contextual similarity matrix and the non-linear combination of relevance scores from query-based similarity vectors. The proposed late fusion approach is evaluated in the multimedia retrieval task, by applying it to two multimedia collections, namely the WIKI11 and IAPR-TC12. The experimental results indicate its superiority over the baseline method in terms of Mean Average Precision for both considered datasets.

Files (923.1 kB)
Name Size
923.1 kB Download
Views 50
Downloads 115
Data volume 106.2 MB
Unique views 47
Unique downloads 107


Cite as