Published June 7, 2022 | Version v2

A Gaze-Driven Digital Interface for Musical Expression Based on Real-time Physical Modelling Synthesis

  • 1. Aalborg University

Description

Individuals with severely limited physical function such as ALS sufferers are unable to engage in conventional music-making activities, as their bodily capabilities are often limited to eye movements. The rise of modern eye-tracking cameras has led to the development of augmented digital interfaces that can allow these individuals to compose music using only their gaze. This paper presents a gaze-controlled digital interface for musical expression and performance using a real-time physical model of a xylophone. The interface was designed to work with a basic Tobii eye-tracker and a scalable, open-source framework was built using the JUCE programming environment. A usability evaluation was carried out with nine convenience-sampled participants. Whilst the interface was found to be a feasible means for gaze-driven music performance our qualitative results indicate that the utility of the interface can be enhanced by expanding the possibilities for expressive control over the physical model. Potential usability improvements include a more robust gaze calibration method, as well as a redesigned graphical interface that is friendlier to individuals lacking musical training. Overall, we see this work as a step towards accessible and inclusive musical performance interfaces for those with major physical limitations.

Files

60.pdf

Files (457.4 kB)

Name Size
md5:05861d1bc8d986436aa1722f8b5c4c30
457.4 kB Preview Download