Conference paper Open Access

ObjectGraphs: Using Objects and a Graph Convolutional Network for the Bottom-up Recognition and Explanation of Events in Video

Gkalelis, Nikolaos; Goulas, Andreas; Galanopoulos, Damianos; Mezaris, Vasileios

In this paper a novel bottom-up video event recognition approach is proposed, ObjectGraphs, which utilizes a rich frame representation and the relations between objects within each frame. Following the application of an object detector (OD) on the frames, graphs are used to model the object relations and a graph convolutional network (GCN) is utilized to perform reasoning on the graphs. The resulting object-based frame-level features are then forwarded to a long short-term memory (LSTM) network for video event recognition. Moreover, the weighted in-degrees (WiDs) derived from the graph’s adjacency matrix at frame level are used for identifying the objects that were considered most (or least) salient for event recognition and contributed the most (or least) to the final event recognition decision, thus providing an explanation for the latter. The experimental results show that the proposed method achieves state-of-the-art performance on the publicly available FCVID and YLI-MED datasets. Source code for our ObjectGraphs method is made publicly available at: https://github.com/bmezaris/ObjectGraphs

Files (4.2 MB)
Name Size
camready_with_header.pdf
md5:0852b6e050198424a8dad21389401c60
4.2 MB Download
18
14
views
downloads
All versions This version
Views 1818
Downloads 1414
Data volume 58.4 MB58.4 MB
Unique views 1616
Unique downloads 1212

Share

Cite as