5006039
doi
10.1007/s11263-021-01443-1
oai:zenodo.org:5006039
user-ai4media
user-eu
Liviu-Daniel Ştefan
University Politehnica of Bucharest
Bogdan Ionescu
University Politehnica of Bucharest
Ngoc Q. K. Duong
InterDigital Paris
Claire-Héléne Demarty
InterDigital Paris
Mats Sjöberg
CSC - IT Center for Science Ltd.
Visual Interestingness Prediction: A Benchmark Framework and Literature Review
Mihai Gabriel Constantin
University Politehnica of Bucharest
info:eu-repo/semantics/openAccess
Creative Commons Attribution 4.0 International
https://creativecommons.org/licenses/by/4.0/legalcode
Media Interestingness
Interestingness
Benchmarking
<p>In this paper, we report on the creation of a publicly available, common evaluation framework for image and video visual interestingness prediction. We propose a robust data set, the Interestingness10k, with 9831 images and more than 4 h of video, interestigness scores determined based on more than 1M pair-wise annotations of 800 trusted annotators, some pre-computed multi-modal descriptors, and 192 system output results as baselines. The data were validated extensively during the 2016–2017 MediaEval benchmark campaigns. We provide an in-depth analysis of the crucial components of visual interestingness prediction algorithms by reviewing the capabilities and the evolution of the MediaEval benchmark systems, as well as of prominent systems from the literature. We discuss overall trends, influence of the employed features and techniques, generalization capabilities and the reliability of results. We also discuss the possibility of going beyond state-of-the-art performance via an automatic, ad-hoc system fusion, and propose a deep MLP-based architecture that outperforms the current state-of-the-art systems by a large margin. Finally, we provide the most important lessons learned and insights gained.</p>
Zenodo
2021-02-22
info:eu-repo/semantics/article
5006038
user-ai4media
user-eu
award_title=A European Excellence Centre for Media, Society and Democracy; award_number=951911; award_identifiers_scheme=url; award_identifiers_identifier=https://cordis.europa.eu/projects/951911; funder_id=00k4n6c32; funder_name=European Commission;
1624283294.827092
18510966
md5:e42632295167521ff1f2aaecb277acc8
https://zenodo.org/records/5006039/files/UPB-IJCV2021.pdf
public
International Journal of Computer Vision
129
1526–1550
2021-02-22