Towards Musically Informed Evaluation of Piano Transcription Models

We provide here the evaluation set employed in our experiments described in "Towards Musically Informed Evaluation of Piano Transcription Models", published in the Proceedings of the 25th International Society for Music Information Retrieval Conference (ISMIR), San Francisco, United States, 2024.

In this work, we demonstrate musically informed piano transcription metrics using transcriptions derived from three state-of-the-art transcriptions ([1], [2], [3]). To this end, we create an evaluation set that includes (1) a subset of the original audio recordings from the MAESTRO dataset [1], (2) a re-recorded version that subset, and (3) a perturbed version of recordings from both (1) and (2). In this data repository, we provide components (2) and (3).

[1] Curtis Hawthorne, Andriy Stasyuk, Adam Roberts, Ian Simon, Cheng-Zhi Anna Huang, Sander Dieleman, Erich Elsen, Jesse Engel, and Douglas Eck, “Enabling factorized piano music modeling and generation with the MAESTRO dataset,” in International Conference on Learning Representations, 2019.  

[2] Qiuqiang Kong, Bochen Li, Xuchen Song, Yuan Wan, and Yuxan Wang, “High-resolution piano transcription with pedals by regressing onset and offset times,” IEEE/ACM Transactions on Audio, Speech and Language Processing, vol. 29, pp. 3707–3717, 2021.  

[3] Curtis Hawthorne, Ian Simon, Rigel Swavely, Ethan Manilow, and Jesse Engel. “Sequence-to-sequence piano transcription with transformers,” in Proceedings of the 22nd International Society for Music Information Retrieval Conference, ISMIR 2021.