Dataset Open Access

Extension of the Action Verb Corpus (AVCext)

Matthias Hirschmanner; Stephanie Gross; Brigitte Krenn; Friedrich Neubarth; Martin Trapp; Michael Zillich; Markus Vincze

Extension of the Action Verb Corpus (AVCext)

The extension to the Action Verb Corpus consists of 41 recordings conducted by 2 users experienced with the system performing the same three actions as in AVC — take (208 instances), put (208 instances), and push (91 instances). The actions were performed without any instructions. The focus of the extension is to facilitate action recognition. In comparison to AVC, no speech-related information is annotated in the extension dataset. The other ELAN annotations available with AVC are also available for the extension dataset. Additionally, the actions are annotated in two degrees of granularity. Coarse labels are: take, put and push. Fine labels split the motion into more granular motion primitives: reach, grab, moveObject, and place. These annotations are available as eaf (ELAN) files, csv files and as two separate columns in the Merged files. Details about the collected data can be found in Matthias Hirschmanner, Stephanie Gross, Brigitte Krenn, Friedrich Neubarth, Martin Trapp and Markus Vincze: Extension of the Action Verb Corpus for Supervised Learning. ARW 2018.

The dataset consists of the following information:

  • the merged output of the hand and object trackers (AVCExtension_Merged.zip, one csv file per episode/recording)
    • HandID: 0 right, 1 left
    • FingerID: 0 thumb, 1 index, 2 middle, 3 ring, 4 pinky
    • BoneID: 0 metacarpal, 1 proximal, 2 intermediate, 3 distal
  • the output of the object trackers, including the object poses and their reliability estimate calculated by the object tracker, whether an object is touched by or is in the hand of the instructor and whether the object touches the table, for the coordinate system applied see picture "coord_system.png (AVCExtension_Objects.zip, one csv file per episode/recording),
  • the videos from Leap Motion showing the hand movements and objects
    (AVCExtension_video_libm.zip, one avi file per episode/recording),
  • animation of the merged hand and object tracking
    (AVCExtension_video_schematic.zip, one avi file per episode/recording),
  • the following list of annotations synchronized with the real-time animation of the hand and object tracking (available as ELAN files AVCExtension_Annotations_eaf.zip, and csv files AVCExtension_Annotations_csv.zip, one file per episode/recording)
    • information which object is currently moved, and where it is moved to (automatically annotated),
    • information whether a hand touches a particular object (manually annotated),
    • information whether a particular object touches the ground/table (automatically annotated),
    • coarse-grained annotation: take, put, push (manually annotated),
    • fine-grained annotation: reach, grab, moveObject, and place (manually annotated),
    • position of the objects in the scene (automatically calculated from output of object tracker)

Acknowledgments

Corpus creation and annotation was supported by the WWTF project RALLI. The dataset was recorded at ACIN, TUW.

Files (403.4 MB)
Name Size
AVCExtension_Annotations_csv.zip
md5:cb578b680d5667d466dd453726f9699c
45.8 kB Download
AVCExtension_Annotations_eaf.zip
md5:bc52fa5531460e3755f2c7d163065b09
125.6 kB Download
AVCExtension_Merged.zip
md5:424e09049e55154aa5a53ad84db7b27c
112.6 MB Download
AVCExtension_Objects.zip
md5:88e9b7cbbc125eb0f99f68071deab24a
2.7 MB Download
AVCExtension_video_libm.zip
md5:4b18e0e52143844511026c19334e6889
186.9 MB Download
AVCExtension_video_schematic.zip
md5:d8c31214eeb59c37749428d36ca63b83
100.8 MB Download
coord_system.png
md5:ee6685811130622dfb7ef7d22a0c88e9
207.5 kB Download
15
33
views
downloads
All versions This version
Views 1515
Downloads 3333
Data volume 1.2 GB1.2 GB
Unique views 1010
Unique downloads 66

Share

Cite as