Millimeter-wave Multi-View Radar (MMVR) Dataset

Introduction

The Millimeter-wave Multi-View Radar (MMVR) dataset comprises 345K multi-view radar frames collected from 25 human subjects over 6 different rooms. It includes 446K annotated bounding boxes/segmentation instances, and 7.59 million annotated keypoints to support three perception tasks: object detection, pose estimation, and instance segmentation in the image plane.

MMVR consists of 35 data sessions, each stored in a data folder with the format dxsy, where x represents the day and y the session index of that day when the data were collected.

Within each data folder, data frames are grouped into 395 non-overlapping one-minute data segments. Each data segment is stored in a folder named using a three-digit, zero-filled convention based on the chronological order. For example, the first one-minute data segment is saved in the folder 000, while the second one-minute data segment is stored in the folder 001.

Within each data segment folder, each data frame consists of 5 NPZ files: meta, radar, bounding boxes (bbox), keypoints (pose) and segmentation masks (mask). Each data frame is named using a five-digit, zero-filled convention based on the chronological order within the one-minute data segment.

Root/
├── d1s1/
│   └── ...
├── d1s2/
│   ├── 000/
│   │   ├── 00000_meta.npz          ... Meta info
│   │   ├── 00000_radar.npz         ... Horizontal/Vertical heatmaps
│   │   ├── 00000_bbox.npz          ... 2D Bounding Boxes
│   │   ├── 00000_pose.npz          ... 2D keypoints
│   │   ├── 00000_mask.npz          ... 2D Segmentation masks
│   │   ├── 00001_meta.npz         
│   │   ├── 00001_radar.npz     
│   │   ├── 00001_bbox.npz      
│   │   ├── 00001_pose.npz      
│   │   ├── 00001_mask.npz      
│   │   .
│   │   ├── 00899_meta.npz          
│   │   ├── 00899_radar.npz         
│   │   ├── 00899_bbox.npz          
│   │   ├── 00899_pose.npz          
│   │   └── 00899_mask.npz          
│   └── 001/
│       ├── 00000_meta.npz          
│       ├── 00000_radar.npz        
│       ├── 00000_bbox.npz         
│       ├── 00000_pose.npz         
│       ├── 00000_mask.npz       
.       └── ...
├── d9s5/
│   └── ... 
└── d9s6/
    └── ...

At a Glance

• • The size of the unzipped dataset is ~81.8 GB
  • The MMVR dataset consists of 345K data frames with each frame including 5 NPZ files: meta, radar, bounding boxes (bbox), keypoints (pose) and segmentation masks (mask).
  • The MMVR dataset is divided into four smaller chunks to facilitate easier downloads: three chunks of approximately 20 GB each and one chunk of 12 GB.
    • P1.zip (24.2 GB):
      • README.md and its related figure folder (figs)
      • load_sample.ipynb: a python snippet to load and visualize a data frame
      • data_split.npz: an npz file containing the predefined data splits (S1 or S2) for training, validation, and test sets of data segments under P1 and P2.
      • all data frames under P1 (d1s1, d1s2, d2s2, d3s1, d3s2, d4s1)
    • P2_00.zip (20.4 GB): all data frames in d5s1 – d5s6 and d6s1 – d6s6
    • P2_01.zip (21.6 GB): all data frames in d7s1 – d7s5 and d8s1 – d8s6
    • P2_02.zip (11.7 GB): all data frames in d9s1 – d9s6

Citation

If you use the MMVR dataset in your research, please cite our contribution:

@inproceedings{MMVR2024, 
   title={MMVR: Millimeter-wave Multi-View Radar Dataset and Benchmark for Indoor Perception}, 
   author={M. Mahbubur Rahman and Ryoma Yataka and Sorachi Kato and Pu Perry Wang and Peizhao Li and Adriano Cardace and Petros Boufounos}, 
   booktitle={Proceedings of European Conference on Computer Vision (ECCV)}, 
   pages={}, 
   year={2024}
}

License

The MMVR dataset is released under CC-BY-SA-4.0 license.

All data:

Created by Mitsubishi Electric Research Laboratories (MERL), 2023
 
SPDX-License-Identifier: CC-BY-SA-4.0