Representation-Plurality in Multi-Touch Mobile Visual Programming for Music

Multi-touch mobile devices provide a fresh paradigm for interactions, as well as a platform for building rich musical applications. This paper presents a multi-touch mobile programming environment that supports the exploration of different representations in visual programming for music and audio interfaces. Using a common flow-based visual programming vocabulary, we implemented a system based on the urMus platform that explores three types of touch-based interaction representations: a text-based menu representation, a graphical icon-based representation, and a novel multi-touch gesture-based representation. We illustrated their use on interface design for musical controllers.