AudioEar: Single-View Ear Reconstruction for Personalized Spatial Audio

Abstract

We introduce AudioEar3D, a high-quality 3D ear dataset consisting of 112 point cloud ear scans with RGB images, to benchmark the ear reconstruction task. We further collect a 2D ear dataset composed of 2,000 images, each one with manual annotation of occlusion and 55 landmarks, named AudioEar2D. To our knowledge, both datasets have the largest scale and best quality of their kinds for public use.

Usage

The code is publicly available at https://github.com/seanywang0408/AudioEar.

The file organization of AudioEar3D is as following:

AudioEar3D

    ├── 001

        # left ear data

        ├── left.jpg # processed RGB image of left ear

        ├── left.ply # processed point cloud of left ear in canonical pose (frontal view is negative-X and upper view is positive-Z)

        ├── left.json # 56 landmark annotations of image

        ├── mask_left.jpg # mask generated by the outer landmarks

        ├── masked_left.jpg # exclude background in left.jpg using mask_left.jpg

        ├── masked_left.png # exclude background in left.jpg using mask_left.jpg, but with four channels of RGB-A

        # right ear data

        ├── right.jpg

        ...

        ├── masked_right.png

    ├── 002

    ...

    ├── 056

The file organization of AudioEar2D is as following:

AudioEar2D

    ├── 00000.png # processed ear image

    ├── 00000.json # landmark annotations

    ...

    ├── 69985.png # the index is aligned with the data source FFHQ.

    ├── 69985.json

Citation

If you find this project useful, currently please cite the paper as:

Xiaoyang Huang, Yanjun Wang, Yang Liu, Bingbing Ni, Wenjun Zhang, Jinxian Liu, Teng Li. "AudioEar: Single-View Ear Reconstruction for Personalized Spatial Audio". arXiv preprint arXiv:2301.12613, 2023.

or using bibtex:

@article{huang2023audioear,
  title={AudioEar: Single-View Ear Reconstruction for Personalized Spatial Audio},
  author={Huang, Xiaoyang and Wang, Yanjun and Liu, Yang and Ni, Bingbing and Zhang Wenjun and Liu Jinxian and Li, Teng},
  journal={arXiv preprint arXiv:2301.12613},
  year={2023}
}

License

The dataset is licensed under Creative Commons Attribution 4.0 International (CC BY 4.0). 

The code is under Apache-2.0 License.

Mirror Link

We recommend users to download the data from Zenodo official link. However, if you find any downloading problem, you can also use this mirror link from Google Drive.

Changelog

v1.0: Initial repository of AudioEar3D and AudioEar2D.