OpenPack: Public multi-modal dataset for packaging work recognition in logistics domain

OpenPack is an open-access logistics dataset for human activity recognition, which contains human movement and package information from 16 subjects in four scenarios. Human movement information is subdivided into three types of data, acceleration, physiological, and depth-sensing. The package information includes the size and number of items included in each packaging job. 

In the "Humanware laboratory" at IST Osaka University, with the supervision of industrial engineers, an experiment to mimic logistic center labor was designed. 12 workers with previous packaging experience and 4 without experience performed a set of packaging tasks according to an instruction manual from a real-life logistics center. During the different scenarios, subjects were recorded while performing packing operations using Lidar, Kinect, and Realsense depth sensors while wearing 4 ATR IMU devices and 2 Empatica E4 wearable sensors. Besides sensor data, this dataset contains timestamp information collected from the hand terminal used to register product, packet, and address label codes as well as package details that can be useful to relate operations to specific packages.

The 4 different scenarios include; sequential packing, worker-decided sequence changes, pre-ordered item packing, and time-sensitive stressors. Each of the subjects performed 20 packing jobs in 5 work sessions for a total of 100 packing jobs. 53+ hours of packaging operations have been labeled into 10 global operation classes and 16 sub-action classes for this dataset. Action classes are not unique to each operation but may only appear in one or two operations. 

You can find information on how to use this dataset at: https://open-pack.github.io/. For details on how this dataset was collected please check the following publication "OpenPack: A Large-Scale Dataset for Recognizing Packaging Works in IoT-Enabled Logistic Environments" 10.1109/PerCom59722.2024.10494448.

Full Dataset

In this repository, the data and label files are contained in separate files for each worker. Each worker's file contains; IMU, E4, 2d keypoint, 3d keypoint, annotation, and system-related data.

Preprocessed Dataset (IMU with operation and action Labels)

We have received many comments that it was difficult to combine multiple workers' IMU and annotation data. Therefore, we have created several CSV files containing the four IMU's sensor data and the operation labels in a single file. These files are now included as "imu-with-operation-action-labels.zip". 

Preprocessed Dataset (Kinect 2D and 3D keypoint data with operation and action Labels)

We have received several requests for a preprocessed dataset containing only specific types of keypoint data with its assigned operation and action labels. Two new preprocessed files have been added for 2D and 3D keypoint data extracted from the frontal view Kinect camera. These files are:

"kinect-2d-kpt-with-operation-action-labels.zip", and

"kinect-3d-kpt-with-operation-action-labels.zip".

Work is continuously being done to update and improve this dataset. When downloading and using this dataset please verify that the version is up to date with the latest release. The latest release [1.1.0] was uploaded on 24/04/2024. 

Changes LOG:

• v1.0.0: Add tutorial preprocessed dataset for IMU data with operation labels.
• v1.1.0: Update preprocessed datasets. (Include Kinect 2d and 3d keypoint data with Operation and action labels)

We hosted an activity recognition competition using this dataset (OpenPack v0.3.x) awarded at a PerCom 2023 Workshop! The task was very simple: Recognize 10 work operations from the OpenPack dataset. You can refer to this website for coding materials relevant to this dataset. https://open-pack.github.io/challenge2022