From Improvised Movement to Musical Improvisation - Using Machine-Learning to Create Personalized Instruments for Dancers

The paper presents the development and preliminary evaluation of a machine-learning-based digital instrument that translates a dancer's bodily movements into music. It is trained on motion-capture and audio recordings of professional dancers improvising solo to music, thereby learning cross-modal correspondences between movement and sound. In performance, a dancer can then use the instrument as a highly personalized tool for generating music through bodily gestures. A transdisciplinary team of machine-learning researchers and dancers leads the development and evaluation, following a practice-led approach aligned with the dancers' artistic interests. This includes selecting the movement and musical material for training and testing, assessing the instrument's creative usability, and integrating it into rehearsals and the creation of new performance works.