Music Everywhere - Augmented Reality Piano Improvisation Learning System

This paper describes the design and implementation of an augmented reality (AR) piano learning tool that uses a Microsoft HoloLens and a MIDI-over-Bluetooth-enabled electric piano. The tool presents a unique visual interface—a "mirror key overlay" approach—fitted for the AR environment, and opens up the possibility of on-instrument learning experiences. The curriculum focuses on teaching improvisation in blues, rock, jazz and classical genres. Users at the piano engage with interactive lessons, watch virtual hand demonstrations, see and hear example improvisations, and play their own solos and accompaniment along with AR-projected virtual musicians. The tool aims to be entertaining yet also effective in teaching core musical concepts.