SUN Horizon Europe Project

Part of EU Open Research Repository
Published May 27, 2024 | Version v1
Journal article Open

Evaluating Performance and Trends in Interactive Video Retrieval: Insights From the 12th VBS Competition

Creators

  • 1. Istituto di Scienza e Tecnologie dell'Informazione Alessandro Faedo Consiglio Nazionale delle Ricerche
  • 2. University of Basel
  • 3. ROR icon Joanneum Research
  • 4. ROR icon Istituto di Scienza e Tecnologie dell'Informazione "Alessandro Faedo"
  • 5. ROR icon National Research Council
  • 6. ROR icon Dublin City University
  • 7. ROR icon HTW Berlin - University of Applied Sciences
  • 8. ROR icon Wuhan University
  • 9. Charles University
  • 10. ROR icon Graz University of Technology
  • 11. ROR icon Singapore Management University
  • 12. University of Zurich
  • 13. ROR icon University of Klagenfurt
  • 14. ROR icon Vietnam National University Ho Chi Minh City
  • 15. ROR icon Centre for Research and Technology Hellas

Description

This paper conducts a thorough examination of the 12th Video Browser Showdown (VBS) competition, a well-established international benchmarking campaign for interactive video search systems. 
The annual VBS competition has witnessed a steep rise in the popularity of multimodal embedding-based approaches in interactive video retrieval. Most of the thirteen systems participating in VBS 2023 utilized a CLIP-based cross-modal search model, allowing the specification of free-form text queries to search visual content.  This shared emphasis on joint embedding models contributed to balanced performance across various teams. However, the distinguishing factors of the top-performing teams included the adept combination of multiple models and search modes, along with the capabilities of interactive interfaces to facilitate and refine the search process.
Our work provides an overview of the state-of-the-art approaches employed by the participating systems and conducts a thorough analysis of their search logs, which record user interactions and results of their queries for each task. Our comprehensive examination of the VBS competition offers assessments of the effectiveness of the retrieval models, browsing efficiency, and user query patterns. Additionally, it provides valuable insights into the evolving landscape of interactive video retrieval and its future challenges.

Files

Evaluating_Performance_and_Trends_in_Interactive_Video_Retrieval_Insights_From_the_12th_VBS_Competition.pdf

Files (3.8 MB)

Additional details

Funding

European Commission
AI4Media – A European Excellence Centre for Media, Society and Democracy 951911
European Commission
SUN – Social and hUman ceNtered XR 101092612
European Commission
XRECO – XR mEdia eCOsystem 101070250
Swiss National Science Foundation
Participatory Knowledge Practices in Analogue and Digital Image Archives 193788
Swiss National Science Foundation
MediaGraph 202125
Vingroup (Vietnam)
VINIF.2019.DA19) VINIF.2019.DA19)
Austrian Research Promotion Agency
886205 886205
Czech Science Foundation
22-21696S 22-21696S
FWF Austrian Science Fund
P 32010-N38 P 32010-N38
National Natural Science Foundation of China
U1903214 U1903214
National Natural Science Foundation of China
62372339 62372339
National Natural Science Foundation of China
61876135 61876135
Science Foundation Ireland
18/CRT/6223 18/CRT/6223