HAND Skills demOnstrated by Multi-subjEcts (HANDSOME) Dataset

The HANDSOME (HAND Skills demOnstrated by Multi-subjEcts) dataset is designed to provide reliable hands and objects motion data during human demonstrations of manual activities. This dataset was originally collected to study interactions between hands and objects in various contexts and automatically map the resulting task representations into robot plans. However, it can be utilized for any application requiring hands and objects detection from RGB video.

The setup involved an RGB camera (Intel RealSense D435i) positioned in a top-down (bird's eye) view, with the image plane aligned parallel to the working plane.
To enable robust detection of the 3D pose of objects and hands, we employed a marker-based detection system. ArUco markers were attached to the back of the hand and strategically positioned on the objects, preserving natural movements during manipulation.
We involved 10 participants, comprising 5 males and 5 females with an average age of 28.4 +/- 2.4 years. Among them, 8 were right-handed and 2 were left-handed. We asked subjects to perform both unimanual and bimanual activities, for a total of 400 recordings, in two different contexts (kitchen and workshop).

The whole experimental procedure was carried out in accordance with the Declaration of Helsinki and the protocol was approved by the ethics committee azienda sanitaria locale (ASL) Genovese N.3 (Protocol IIT_HRII_ERGOLEAN 156/2020).