10.5281/zenodo.4963588
https://zenodo.org/records/4963588
oai:zenodo.org:4963588
Gkalelis, Nikolaos
Nikolaos
Gkalelis
CERTH-ITI
Goulas, Andreas
Andreas
Goulas
CERTH-ITI
Galanopoulos, Damianos
Damianos
Galanopoulos
CERTH-ITI
Mezaris, Vasileios
Vasileios
Mezaris
CERTH-ITI
ObjectGraphs: Using Objects and a Graph Convolutional Network for the Bottom-up Recognition and Explanation of Events in Video
Zenodo
2021
video analysis
video event recognition
graph convolutional network
long short-term memory
object detection
explainable artificial intelligence
2021-06-16
10.5281/zenodo.4963587
https://zenodo.org/communities/h2020-mirror-project
https://zenodo.org/communities/ai4media
https://zenodo.org/communities/eu
Creative Commons Attribution 4.0 International
In this paper a novel bottom-up video event recognition approach is proposed, ObjectGraphs, which utilizes a rich frame representation and the relations between objects within each frame. Following the application of an object detector (OD) on the frames, graphs are used to model the object relations and a graph convolutional network (GCN) is utilized to perform reasoning on the graphs. The resulting object-based frame-level features are then forwarded to a long short-term memory (LSTM) network for video event recognition. Moreover, the weighted in-degrees (WiDs) derived from the graph’s adjacency matrix at frame level are used for identifying the objects that were considered most (or least) salient for event recognition and contributed the most (or least) to the final event recognition decision, thus providing an explanation for the latter. The experimental results show that the proposed method achieves state-of-the-art performance on the publicly available FCVID and YLI-MED datasets. Source code for our ObjectGraphs method is made publicly available at: https://github.com/bmezaris/ObjectGraphs
European Commission
10.13039/501100000780
832921
Migration-Related Risks caused by misconceptions of Opportunities and Requirement
European Commission
10.13039/501100000780
951911
A European Excellence Centre for Media, Society and Democracy