Hi everyone, I’m Andrea Guidi and the paper I’m about to present is called
Design for Auditory Imagery: altering instruments to explore performer fluency.
This research is the result of shared work with Fabio Morreale and Andrew McPherson.
Auditory imagery is often described as
“The ability to hear or recreate sounds in the mind
even when no audible sound is present”
It is an essential skill developed by professional musicians and a fundamental component of music
execution where players do not consciously think about the instrument.
Auditory imagery allows musicians to formulate auditory expectations of what the sound will be,
perform detailed sensory-motor tasks on their instrument to achieve it,
and eventually, adjust their playing by comparing those expectations against the auditory feedback
coming from the instrument.
This study introduces the idea that auditory imagery is essential in the design of interfaces
that can be played by professional musicians using their existing skills.
Allowing musicians to use their skills to play a new interface can
speed up the learning process,
facilitate the adoption of the interface,
facilitate the creation of new repertoires for that interface.
We propose that auditory imagery is a critical element
in the design of those interfaces
that leverage on musician’s existing skills
to play new sounds.
That’s because if players cannot Imagine a sound, then they cannot play it.
A lack of imagery leads to a lack of motor program to perform on the instrument.
A motor program can be described as an abstract representation of the movements
needed to perform a certain gesture on an instrument.
For instance, plucking a string.
To verify this principle, we conducted a study divided in two parts
In part one we asked 12 violinists trained in the western classical music repertoire
to play quarter tones and semitones
The selected violinists had no training in quarter tones and they were not familiar with
micro intervals.
We asked them to play quarter tones to evaluate
their pitch accuracy while playing sounds for which they did not have auditory imagery.
Violinists were asked to play semitones and quarter tones notes with their personal violin
in two separate conditions: reading notational stimuli and listening to audio stimuli.
The audio section was meant to discriminate the effect of notation coming from reading the
unfamiliar symbols describing quarter tones.
This is an example of a notation
showing a semitone stimulus
And this i is an example of a notation
showing a quarter tone sharp stimulus
This is an audio stimulus example playing a semitone
This is an audio stimulus example playing a quarter tone
We audio-recorded violinists performances and we measured the time interval between
participants reading each notation on a display,
and the beginning of their performance.
We also measured their pitch accuracy for each note.
For the pitch analysis, we segmented each note into
start (100 ms following the attack phase)
mean (the time interval following the start of the note up to the 100 ms before the end of the note)
and end (the last 100 ms before the violin bow detached from the strings of the violin).
Results show that
quarter tones were performed with less pitch accuracy and with a longer time response than semitones.
Violinists had the right mapping precision, the right auditory feedback, and a familiar instrument.
Still, they found difficult playing quarter tones.
We propose that they struggled to play quarter tones because they did not have the right
auditory imagery and so the right motor program to play them.
A lack of imagery prevented them to play sounds unfamiliar to their training even on
an interface they practised for years.
In the second part of the study, we asked the same 12 violinists to perform on a retuned violin.
The violin was tuned in fourths (starting from A3) instead of fifths.
We re-tuned a violin
to look at the consequence that the instrument modification
has on violinists playing
We observed how violinists’ tackled the challenge of playing
the modified version of a familiar interface (the violin) in two separate conditions
In the first condition, violinists played reading concert notations.
This type of notation suggest
the right auditory feedback but do not indicates the right gestures
to perform on the re-tuned violin to play the notes.
In the second condition,
notation is transposed to match the violin re-tuning.
As a result, the notation does not visually suggest
the right auditory feedback. However,
it indicates the gestures to execute
to achieve a melody on the transposed violin.
This second part of the study was designed to reply to the following question:
when playing on a modified instrument,
is it more important being able to imagine the right auditory feedback (as in the concert notation)?
or the right gestures to perform (as in the transposed notation)
In the concert notation condition,
participants performed 770 extra notes
resulting in longer performances.
These notes are repeated notes
or wrong notes
performed in the attempt of finding
the right finger position
on the re-tuned violin.
In the transposed notation condition,
we can observe a significantly reduced number of
extra notes (49 versus 770).
In this latter condition, executions had both a number of
notes and a duration
more coherent with the task’s indications.
This is an audio recording of a participant reading transposed notation.
This is an audio recording of a participant reading concert notation.
We propose that transposed notation worked so well because it let musicians use their
familiar auditory imagery, which in turn, is connected to the actions needed to perform the notation.
Even if the resulting sound was not the one they imagined to be, their fluency improved,
and so the fluency of their performance.
This suggests that having access to an existing auditory imagery and motor programs is more
critical than perceiving the correct auditory feedback.
In the transposed notation scenario, the imagery was deliberately incorrect.
The transposed notation did not produce the sounds that it specifies.
Still, it worked better to use
a notation that specifies the correct motor actions and that produces a “wrong” but
familiar auditory result (transposed notation) than to have a notation specifying
a “correct” but unfamiliar auditory feedback (concert notation).
We propose that a lack of auditory imagery
constitutes an important design constraint
for interfaces that
allow musicians to perform new or unfamiliar sounds,
aim to leverage on existing performer’s skills to make the instrument playable
in a short amount of time,
aim to allow the kind of embodied performance that is generally available on traditional instruments.
In the design of such interfaces, the following design question becomes essential:
is there an intrinsic link between the particular interface that is being designed,
and the existing imagination
of performers that will play the interface
plus the execution techniques they already know?
If players cannot imagine the sounds produced by the instrument, then they cannot play them.
Unless they decide to invest time in a training process to develop an imagery
which would be specific for that new interface.
Thanks for listening