Forearm Ultrasound Dataset for Hand Gesture Classification and Finger Joint Angle Estimation

Forearm B-mode ultrasound recordings with time-synchronized finger metacarpophalangeal (MCP) joint angles, for research on hand gesture classification and continuous finger joint angle estimation from forearm ultrasound (sonomyography). This is the data underlying:

Keshav Bimbraw, Christopher J. Nycz, Matthew Schueler, Ziming Zhang, and Haichong K. Zhang, "Simultaneous Estimation of Hand Configurations and Finger Joint Angles Using Forearm Ultrasound," IEEE Transactions on Medical Robotics and Bionics, vol. 5, no. 1, Feb. 2023. doi:10.1109/TMRB.2023.3237774

Human-subjects data collection was approved by the Worcester Polytechnic Institute Institutional Review Board (IRB-21-0452).

Participants, gestures, and speeds

• 6 participants (Test_Subject_1 … Test_Subject_6), one ZIP archive per subject.
• 11 hand configurations (activities of daily living): IndFlex (index finger flexion), MidFlex (middle), RinFlex (ring), PinFlex (pinky), IndPinch (index to thumb), IndMidPinch, IndMidRinPinch, AllPinch (all fingertips to thumb), MidRinPinch, Fist, and Hook (interphalangeal flexion with the MCP joint held). Subjects alternated between an open (rest) hand and each configuration.
• 3 speeds: Slow (0.5 Hz), Medium (1 Hz), and Fast (2 Hz), cued by audio tones.
• Each session is 56 seconds at 25 Hz = 1400 ultrasound frames.

Acquisition

Ultrasound — Verasonics Vantage 128 with an L12/5 50 mm linear-array probe mounted transversely on the forearm; post-beamformed, envelope-detected B-mode frames at 25 Hz, each 636 × 256 pixels (44.52 mm depth × 50 mm width), stored at linear (pre-log-compression) scale.

Motion capture — 10-camera Vicon Vantage system at 100 Hz, with markers on the metacarpal and proximal-phalanx ends of the index, middle, ring, and pinky fingers. Each MCP angle is the angle between that finger's metacarpal and proximal-phalanx vectors. A 10 kHz analog trigger emitted by the ultrasound system was recorded by the motion-capture system and used to align each ultrasound frame to the finger angles. The thumb was not instrumented.

File organization

Each Test_Subject_N.zip expands to:

Test_Subject_N/<Speed>_Speed/<Gesture>/data.npz

Each data.npz (NumPy .npz) contains:

• ultrasound — (N, 636, 256) float16 — envelope frames, stored as raw_envelope / us_scale
• us_scale — scalar float32 — multiply the ultrasound array by this to recover the true envelope
• mcp_angles — (N, 4) float32 — MCP flexion angles in degrees for [Index, Middle, Ring, Pinky], one row per frame
• fingers — the four finger names

Loading (Python):

import numpy as np
d = np.load("Test_Subject_1/Slow_Speed/Fist/data.npz")
ultrasound = d["ultrasound"].astype("float32") * d["us_scale"]   # (N, 636, 256) raw envelope
angles = d["mcp_angles"]                                          # (N, 4) degrees

The raw envelope spans a large linear range (up to about 1e7, beyond the float16 range), so each session is divided by a per-session us_scale before casting to float16; multiplying back recovers the envelope losslessly to float16 precision, at roughly half the storage of float32.

Sessions not available

The following (subject, speed, gesture) sessions have no data.npz in this release:

• Test_Subject_4 / Slow_Speed / IndFlex
• Test_Subject_4 / Fast_Speed / Fist
• Test_Subject_5 / Slow_Speed / IndFlex
• Test_Subject_6 / Slow_Speed / IndFlex

In addition, Test_Subject_4 / Fast_Speed / MidFlex contains an incomplete data.npz (ultrasound frames only, and not reliably readable) and should be disregarded.

All 193 remaining sessions contain both the ultrasound frames and the finger angles.

License and citation

Released under the Creative Commons Attribution 4.0 International (CC BY 4.0) license. If you use this dataset, please cite the article above and this Zenodo record.