Dataset of surface EMG recordings from 11 subjects performing single and combination gestures, from "**A Multi-label Classification Approach to Increase Expressivity of EMG-based Gesture Recognition**" by Niklas Smedemark-Margulies, Yunus Bicer, Elifnur Sunger, Stephanie Naufel, Tales Imbiriba, Eugene Tunik, Deniz Erdogmus, and Mathew Yarossi.

For more details and example usage, see the following:

• Paper pdf - https://arxiv.org/pdf/2309.12217.pdf
• Experiment code - https://github.com/neu-spiral/multi-label-emg

Contents

Dataset of single and combination gestures from 11 subjects. 
Subjects participated in 13 experimental blocks.
During each block, they followed visual prompts to perform gestures while also manipulating a joystick.
Surface EMG was recorded from 8 electrodes on the forearm; labels were recorded according to the current visual prompt and the current state of the joystick.

Experiments included the following blocks:

• 1 Calibration block
• 6 Simultaneous-Pulse Combination blocks (3 without feedback, 3 with feedback)
• 6 Hold-Pulse Combination blocks (3 without feedback, 3 with feedback)

The contents of each block type were as follows:

• In the Calibration block, subjects performed 8 repetitions of each of the 4 direction gestures, 2 modifier gestures, and a resting pose.
  Each Calibration trial provided 160 overlapping examples, for a total of: 8 repetitions x 7 gestures x 160 examples = 8960 examples.
• In Simultaneous-Pulse Combination blocks, subjects performed 8 trials of combination gestures, where both components were performed simultaneously.
  Each Simultaneous-Pulse trial provided 240 overlapping examples, for a total of: 8 trials x 240 examples = 1920 examples.
• In Hold-Pulse Combination blocks, subjects performed 28 trials of combination gestures, where 1 gesture component was held while the other was pulsed.
  Each Hold-Pulse trial provided 240 overlapping examples, for a total of: 28 trials x 240 examples = 6720 examples.

A single data example (from any block) corresponds a window 250ms of EMG recorded at 1926Hz (built-in 20–450 Hz bandpass filtering applied).
A 50ms step size was used between each window; note that neighboring data examples are therefore overlapping.

Feedback was provided as follows:

• In blocks with feedback, a model pre-trained on the Calibration data was used to give realtime visual feedback during the trial.
• In blocks without feedback, no model was used, and the visual prompt was the only source of information about the current gesture.

For more details, see the paper.

Labels

Two types of labels are provided: 

• joystick labels were recorded based on the position of the joystick, and are treated as ground-truth.
• visual labels were also recorded based on what prompt was currently being shown to the subject.

For both joystick and visual labels, the following structure applies. Each gesture trial has a two-part label.

The first label component describes the direction gesture, and takes values in {0, 1, 2, 3, 4}, with the following meaning:

• 0 - "Up" (joystick pull)
• 1 - "Down" (joystick push)
• 2 - "Left" (joystick left)
• 3 - "Right" (joystick right)
• 4 - "NoDirection" (absence of a direction gesture; none of the above)

The second label component describes the modifier gesture, and takes values in {0, 1, 2}, with the following meaning:

• 0 - "Pinch" (joystick trigger button)
• 1 - "Thumb" (joystick thumb button)
• 2 - "NoModifier" (absence of a modifier gesture; none of the above)

Examples of Label Structure

Single gestures have labels like (0, 2) indicating ("Up", "NoModifier") or (4, 1) indicating ("NoDirection", "Thumb").

Combination gesture have labels like (0, 0) indicating ("Up", "Pinch") or (2, 1) indicating ("Left", "Thumb").

File layout

Data are provided in Numpy and MATLAB format. Descriptions below apply for both.

Each experimental block is provided in a separate folder.
Within one experimental block, the following files are provided:

• `data.npy` - Raw EMG data, with shape (items, channels, timesteps).
• `joystick_direction_labels.npy` - one-hot joystick direction labels, with shape (items, 5).
• `joystick_modifier_labels.npy` - one-hot joystick modifier labels, with shape (items, 3).
• `visual_direction_labels.npy` - one-hot visual direction labels, with shape (items, 5).
• `visual_modifier_labels.npy` - one-hot visual modifier labels, with shape (items, 3).

Loading data

For example code snippets for loading data, see the associated code repository.