Learning Advanced Skills on New Instruments

When learning a classical instrument, people often eithertake lessons in which an existing body of "technique" is delivered, evolved over generations of performers, or in somecases people will "teach themselves" by watching people playand listening to existing recordings. What does one do witha complex new digital instrument?In this paper I address this question drawing on my experience in learning several very different types of sophisticatedinstruments: the Glove Talk II real-time gesture-to-speechinterface, the Digital Marionette controller for virtual 3Dpuppets, and pianos and keyboards. As the primary userof the first two systems, I have spent hundreds of hourswith Digital Marionette and Glove-Talk II, and thousandsof hours with pianos and keyboards (I continue to work asa professional musician). I will identify some of the underlying principles and approaches that I have observed duringmy learning and playing experience common to these instruments. While typical accounts of users learning new interfaces generally focus on reporting beginner's experiences, forvarious practical reasons, this is fundamentally different byfocusing on the expert's learning experience.