RoHuCAD: Robots and Humans Collaborative Anomaly Detection

RoHuCAD is a dataset of human-robot collaboration in a robotic workshop (check workshop_layout.png). Two robots (collaborative manipulator - cobot, autonomous mobile robot - AMR) assist three human operators in assembly of electronic devices.

There are two 8-min long recordings in the dataset. They mostly follow the same scenario, with slightly different anomalies. The data is in ROS Noetic rosbag format.

Included data 

• RGBD camera data (color + depth)
  • 3 cameras: Intel Realsense D435i
  • color and depth data at 6 frames per second
  • Intrinsic calibration data
  • Extrinsic calibration data (positions and orientations)
• Information about positions of robots
  • AMR: Ez-Wheel SWD® Starter Kit
  • Cobot: Universal Robots UR10e

Annotations

Annotations of specific anomalies are included (CSV file with columns: event_id, tstart, tend, event_type, person_id, camera_id)

• Gestures / poses
  • BENT
  • T-POSE (hands horizontally to the sides)
  • L+R-UP (both hands up)
  • RH-UP (right hand up)
  • LH-UP (left hand up)
  • SQUAT
  • HI-POSE (waving)
• Unsafe behaviour
  • Human in robot working area
  • Standing back to (moving) robot
  • Looking at phone
  • Human in the way of AMR
• Normal activities
  • Assembling/Working
  • Loading/unloading AMR

ROS topics

• /tf
• /tf_static
• /joint_states
• cam_ws2_box
  • /cam_ws2_box/color/camera_info
  • /cam_ws2_box/color/image_raw/compressed
  • /cam_ws2_box/depth_registered/camera_info
  • /cam_ws2_box/depth_registered/image_rect_raw
• cam_ta2_ws2
  • /cam_ta2_ws2/color/camera_info
  • /cam_ta2_ws2/color/image_raw/compressed
  • /cam_ta2_ws2/depth_registered/camera_info
  • /cam_ta2_ws2/depth_registered/image_rect_raw
• cam_ta1_ws2
  • /cam_ta1_ws2/color/camera_info
  • /cam_ta1_ws2/color/image_raw/compressed
  • /cam_ta1_ws2/aligned_depth_to_color/camera_info
  • /cam_ta1_ws2/aligned_depth_to_color/image_raw

Acknowledgement

The work leading to these results has received funding from the European Union’s Horizon Europe research and innovation programme within the ULTIMATE project under the Grant Agreement no 101070162.