Online Symbolic Music Alignment With Offline Reinforcement Learning

Symbolic Music Alignment is the process of matching performed MIDI notes to corresponding score notes. In this paper, we introduce a reinforcement learning (RL)- based online symbolic music alignment technique. The RL agent — an attention-based neural network — itera- tively estimates the current score position from local score and performance contexts. For this symbolic alignment task, environment states can be sampled exhaustively and the reward is dense, rendering a formulation as a simpli- fied offline RL problem straightforward. We evaluate the trained agent in three ways. First, in its capacity to identify correct score positions for sampled test contexts; second, as the core technique of a complete algorithm for symbolic online note-wise alignment; and finally, as a real-time sym- bolic score follower. We further investigate the pitch-based score and performance representations used as the agent's inputs. To this end, we develop a second model, a two- step Dynamic Time Warping (DTW)-based offline align- ment algorithm leveraging the same input representation. The proposed model outperforms a state-of-the-art refer- ence model of offline symbolic music alignment.