Conference paper Open Access

Deep 3D Flow Features for Human Action Recognition

Psaltis, Athanasios; Papadopoulos, Th. Georgios; Daras, Petros

The present work investigates the use of 3D flow information for performing Deep Learning (DL)-based human action recognition. Generally, 3D flow fields include rich and fine-grained information, regarding the motion dynamics of the observed human actions. However, despite the great potentials present, 3D flow has not been widely used, mainly due to challenges related to the efficient modeling of the flow information and the addressing of the respective computational complexity issues. In this paper, different techniques are investigated for incorporating 3D flow information in DL action recognition schemes. In particular, a novel sequence modeling approach is introduced, which combines the advantageous characteristics for spatial correlation estimation of Convolutional Neural Networks (CNNs) with the increased temporal modeling capabilities of Long Short Term Memory (LSTM) models. Additionally, an extended CNN - based deep flow model is proposed that extracts features from both the spatial and temporal domains, by applying 3D convolutions; hence, modeling the action dynamics within consecutive frames. Moreover, for compact and efficient 3D motion feature extraction, the combined use of CNNs with a `flow colorization' approach is adopted. The proposed methods significantly outperform similar DL and hand-crafted 3D flow approaches, and compare favorably with most skeleton-based techniques in the currently most challenging public dataset, namely the NTU RGB-D.

Files (587.6 kB)
Name Size
Deep 3D Flow Features for Human Action Recognition.pdf
md5:b869a188ffd539b17d0f1c800218f23e
587.6 kB Download
32
43
views
downloads
Views 32
Downloads 43
Data volume 25.3 MB
Unique views 26
Unique downloads 37

Share

Cite as