mmWave-based Activity Recognition Dataset

Description:

This mmWave Datasets are used for activity verification. It contains two datasets. The first dataset (FA Dataset) contains 14 common daily activities. This second one (EA Dataset) contains 5 kinds of eating activities. The data are captured by the mmWave radar TI-AWR1642. The dataset can be used by fellow researchers to reproduce the original work or to further explore other machine-learning problems in the domain of mmWave signals.

Format: .png format

Section 1: Device Configuration

• A commodity mmWave radar TI AWR1642, which integrates a 2 × 4 antenna array. The detailed information of it can be found at https://www.ti.com/product/AWR1642#:~:text=The%20AWR1642%20is%20an%20ideal,of%2076%20to%2081%20GHz.
• A TI DCA1000EVM data capture card is used to collect data from the mmWave device and send data to a laptop. The detailed information can be found at https://www.ti.com/tool/DCA1000EVM?keyMatch=DCA1000EVM.
• mmWave radar work at the frequency in the range of 77~81GHz. The sampling rate is fixed at 100 frames per second and each frame has 17 chirps.

Section 2: Data Format

We provide our mmWave data in heatmaps for the two datasets. The data file is in the png format. The details are shown in the following:

FA Dataset

• 2 participants are included in the FA Dataset.
• 14 activities are included in the FA Dataset.
• FA_d_p_i_u_j.png:
  • d represents the data to collect the data.
  • p represents the environment to collect the data.
  • i represents activity type index
  • u represents user id
  • j represents sample index
• Example:
  • FA_20220101_lab_1_2_3 represents the 3rd data sample of user 2 of activity 1 collected in the lab

EA Dataset

• 2 participants are included in the EA Dataset.
• 5 activities are included in the EA Dataset.
• EA_d_p_i_u_j.png:
  • d represents the data to collect the data.
  • p represents the environment to collect the data.
  • i represents the activity type index
  • u represents the user id
  • j represents the sample index

Section 3: Experimental Setup

FA Dataset

• We place the mmWave device on a table with a height of 60cm.
• The participants are asked to perform fitness activity in front of a mmWave device with a distance of 2m.
• The data are collected at an lab with a size of (5.0m×3.0m).

EA Dataset

• We place the mmWave device on a table with a height of 60cm.
• The participants are asked to eat with different utensils (i.e., fork, fork&knife, spoon, chopsticks, bare hand) in front of a mmWave device with a distance of 1m.
• The data are collected at an lab with a size of (5.0m×3.0m).

Section 4: Data Description

• We develop a spatial-temporal heatmap to integrates multiple activity features, including the range of movement, velocity, and time duration of each activity repetition.

 

• We first derive the Doppler-range map of the users’ activity by calculating Range-FFT and Doppler-FFT. Then, we generate the spatial-temporal heatmap by accumulating the velocity of every distance in every Doppler-range map together. Next, we normalize the derived velocity information and present the velocity-distance relationship in time dimension. In this way, we transfer the original instantaneous velocity-distance relationship to a more comprehensive spatial-temporal heatmap which describes the process of a whole activity.

 

• As shown in Figure attached, in each spatial-temporal heatmap, the horizontal axis represents the time duration of an activity repetition while the vertical axis represents the range of movement. The velocity is represented by color.

 

• We create 2 folders to store two dataset respectively. In FA folder, there are 14 subfolders, each contains repetitions from the same fitness activity. In EA folder, there are 5 subfolders, each contains repetitions with different utensils.

14 common daily activities and their corresponding folders

Folder Name	Activity Type	Folder Name	Activity Type
FA1	Crunches	FA8	Squats
FA2	Elbow plank and reach	FA9	Burpees
FA3	Leg raise	FA10	Chest squeezes
FA4	Lunges	FA11	High knees
FA5	Mountain climber	FA12	Side leg raise
FA6	Punches	FA13	Side to side chops
FA7	Push ups	FA14	Turning kicks

 

5 eating activities and their corresponding folders

Folder Name	Activity Type
EA1	Eating with chopsticks
EA2	Eating with fork
EA3	Eating with bare hand
EA4	Eating with fork&knife
EA5	Eating with spoon

 

Section 5: Raw Data and Data Processing Algorithms

• We also provide the mmWave raw data (.mat format) stored in the same folder corresponding to the heatmap datasets. Each .mat file can store one set of activity repetitions (e.g., 4 repetations) from a same user.
  • For example: EA_d_p_i_u_j.mat:
    • d represents the data to collect the data.
    • p represents the environment to collect the data.
    • i represents the activity type index
    • u represents the user id
    • j represents the set index
• We plan to provide the data processing algorithms (heatmap_generation.py) to load the mmWave raw data and generate the corresponding heatmap data.

Section 6: Citations

If your paper is related to our works, please cite our papers as follows.

https://ieeexplore.ieee.org/document/9868878/

Xie, Yucheng, Ruizhe Jiang, Xiaonan Guo, Yan Wang, Jerry Cheng, and Yingying Chen. "mmFit: Low-Effort Personalized Fitness Monitoring Using Millimeter Wave." In 2022 International Conference on Computer Communications and Networks (ICCCN), pp. 1-10. IEEE, 2022.

Bibtex:

@inproceedings{xie2022mmfit,

  title={mmFit: Low-Effort Personalized Fitness Monitoring Using Millimeter Wave},

  author={Xie, Yucheng and Jiang, Ruizhe and Guo, Xiaonan and Wang, Yan and Cheng, Jerry and Chen, Yingying},

  booktitle={2022 International Conference on Computer Communications and Networks (ICCCN)},

  pages={1--10},

  year={2022},

  organization={IEEE}

}

https://www.sciencedirect.com/science/article/abs/pii/S2352648321000532

Xie, Yucheng, Ruizhe Jiang, Xiaonan Guo, Yan Wang, Jerry Cheng, and Yingying Chen. "mmEat: Millimeter wave-enabled environment-invariant eating behavior monitoring." Smart Health 23 (2022): 100236.

Bibtex:

@article{xie2022mmeat,

  title={mmEat: Millimeter wave-enabled environment-invariant eating behavior monitoring},

  author={Xie, Yucheng and Jiang, Ruizhe and Guo, Xiaonan and Wang, Yan and Cheng, Jerry and Chen, Yingying},

  journal={Smart Health},

  volume={23},

  pages={100236},

  year={2022},

  publisher={Elsevier}

}