Human-to-robot Handovers of Cups with Water

If you use this dataset in your work, please cite the following publication (currently under-review in IEEE iROS 2023).

Setup Description:

The experiment is presented as a collaborative task, where the human should help the robot clean the table by handing over the cups, from the rightmost cup to the leftmost, one at a time. The robot receives the cup; in case the cup contains water, it pours the content into the orange bucket, and finally, it places the empty cup in the blue drawer.

Participants stand in front of a table with four identical plastic cups placed in a row, equidistant from each other. These cups differ in content, being two empty and two filled with water almost to the brim, constituting two types of objects to be handover: empty or full. Participants faced a Kinova Gen3 robot fixed to a table with two distinct recipients at the robot side. On the left side of the robot, there is an orange bucket meant to contain water, while the blue drawer on the right stores the empty (or emptied) cups. We adopted a within-subject study design where participants are exposed, in a randomized order, to two conditions associated with the controller used by the robot to complete the task: a neutral motion (NEU) and an expressive motion (GAN). The neutral is a simple PID controller and the expressive is Generative Adversarial Network (GAN) which generates robot trajectories that are human-inspired from a previous dataset of humans handing over cups with water and without.

Our study involved 15 right-handed participants (8 females, 7 males, average of 26.6 (+-6.2) years old) who provided written informed consent. They were all naive regarding the purpose of the experiments and not directly involved in our research. The self-reported level of knowledge in robotics was: 40.0% professional or advanced, 33.3% average, and 26.7% little or none.
360 actions (15 participants x 12 handovers x 2 conditions) were recorded and performed successfully without dropping the cup or spilling the content.

Data Description:

All the data is synchronize using ROS timestamps.

- motion-tracking: Motion Capture data for head, shoulder, and wrist from OptiTrack at 120 Hz + IMU wrist data at 400 Hz. Inside each participant P## folder you will find two other folders P##_neu and P##_gan related to the two interaction conditions where the kinova motion controller changed during the cup pouring and cup placing. Inside each subfolder you will find the following motion tracking data: head.csv, shoulder.csv, wrist.csv, robot.csv (from OptiTrack markers, the robot.csv is the robot's base), imu.csv (from IMU in the wrist), pupil.csv (Pupil ROS node), key.csv (manual labels). All these files have ROS_timestamps that can be used to find the matching frames for each of the sensors. The key.csv are manually picked time flags we marked to define specific moments in the experiment (you can the meaning in the additional notes below).

- eye-tracking_#: Pupil-Labs head-mounted eye-trackers at 120Hz for pupil infra-red cameras, and 30 Hz for forward RGB camera. All 16 participants (P##) are present for both robot motion controllers (neutral NEU, and GAN).

- go_pro_#: GoPro 1080p video of the size view of the Human-to-robot handovers experiments at 60 Hz. Note that there were 16 participants in this experiment but 3 participants did not give permission to make their image public so we removed the following participants videos: P01, P15, P16.