Spatial-Temporal Heatmap Construction Algorithms

Description:

We develop a spatial-temporal heatmap algorithm to extract features from raw mmWave data. The heatmap integrates multiple activity features, including the range of movement, velocity, and time duration of each activity repetition. You can generate the heatmaps as we published in the datasets we shared using this algorithm.

Algorithm description:

• We first perform range-FFT and doppler-FFT signal processing on the raw data to derive distance and velocity measurements, respectively.We normalize the derived velocity information and present the velocity-distance relationship in time dimension.

• To mitigate the environmental interference, we propose an environmental impact mitigation method by filtering out non-moving objects in doppler-range domain.

• To integrate multi-dimensional features including velocity, distance, and temporal information, we propose to construct spatial-temporal heatmaps by accumulate the velocity of every distance in every doppler-range heatmap together as follows: 

                                                                      \(\begin{equation}\label{equ:locations} V_{q,t}=\sum_{p=1}^{D} (I_{p,q,t})\times v_{p,t}, p\in [1, D], q\in [1,R], \end{equation}\)

           where I_p,q,t is the intensity of a frequency response in the doppler-range heatmap, p is the doppler index, q represents the range index, and t is the frame index. v_p,t is the velocity corresponding to a doppler index p in frame t.

• We normalize the derived V q,t and transfer the original instantaneous velocity-distance relationship to a more comprehensive spatial-temporal heatmap which describes the process of a workout as shown in the attachment.

• We determines the 2D window size of each repetition according to its time duration and range of movement in the spatial-temporal heatmap.

For more detailed information about the spatial-temporal heatmap construction algorithm, please refer the following papers:

If your paper is related to our works, please cite our papers as follows.

Xie, Yucheng, Ruizhe Jiang, Xiaonan Guo, Yan Wang, Jerry Cheng, and Yingying Chen. "mmFit: Low-Effort Personalized Fitness Monitoring Using Millimeter Wave." In 2022 International Conference on Computer Communications and Networks (ICCCN), pp. 1-10. IEEE, 2022.

Bibtex:

@inproceedings{xie2022mmfit,

  title={mmFit: Low-Effort Personalized Fitness Monitoring Using Millimeter Wave},

  author={Xie, Yucheng and Jiang, Ruizhe and Guo, Xiaonan and Wang, Yan and Cheng, Jerry and Chen, Yingying},

  booktitle={2022 International Conference on Computer Communications and Networks (ICCCN)},

  pages={1--10},

  year={2022},

  organization={IEEE}

}

Xie, Yucheng, Ruizhe Jiang, Xiaonan Guo, Yan Wang, Jerry Cheng, and Yingying Chen. "mmEat: Millimeter wave-enabled environment-invariant eating behavior monitoring." Smart Health 23 (2022): 100236.

Bibtex:

@article{xie2022mmeat,

  title={mmEat: Millimeter wave-enabled environment-invariant eating behavior monitoring},

  author={Xie, Yucheng and Jiang, Ruizhe and Guo, Xiaonan and Wang, Yan and Cheng, Jerry and Chen, Yingying},

  journal={Smart Health},

  volume={23},

  pages={100236},

  year={2022},

  publisher={Elsevier}

}