Published January 30, 2024 | Version v1
Conference paper Open

An Integrated System for Spatio-Temporal Summarization of 360-degrees Videos

  • 1. ROR icon Centre for Research and Technology Hellas

Description

In this work, we present an integrated system for spatiotemporal summarization of 360-degrees videos. The video summary production mainly involves the detection of salient events and their synopsis into a concise summary. The analysis relies on state-of-the-art methods for saliency detection in 360-degrees video (ATSal and SST-Sal) and video summarization (CA-SUM). It also contains a mechanism that classifies a 360-degrees video based on the use of static or moving camera during recording and decides which saliency detection method will be used, as well as a 2D video production component that is responsible to create a conventional 2D video containing the salient events in the 360-degrees video. Quantitative evaluations using two datasets for 360-degrees video saliency detection (VR-EyeTracking, Sports-360) show the accuracy and positive impact of the developed decision mechanism, and justify our choice to use two different methods for detecting the salient events. A qualitative analysis using content from these datasets, gives further insights about the functionality of the decision mechanism, shows the pros and cons of each used saliency detection method and demonstrates the advanced performance of the trained summarization method against a more conventional approach.

Files

MMM2024_360_zenodo.pdf

Files (7.9 MB)

Name Size Download all
md5:00a08e45cd501b8cbb1bb1ed9c2e18fc
7.9 MB Preview Download

Additional details

Funding

TransMIXR – Ignite the Immersive Media Sector by Enabling New Narrative Visions 101070109
European Commission
AI4Media – A European Excellence Centre for Media, Society and Democracy 951911
European Commission