Video clips show early collaborative play in the making of two inclusive
theatre performances: Oily Cart's Jamboree, 2019
Ellie Griffiths' Sound Symphony, 2017 to 2019.
The motivations behind this research come mainly from my background
in inclusive performance.
The groups shown here, like many others around the world,
are working harder to find better ways of co-producing art
with young neurodiverse audiences.
In the early stages of this kind of work, it can be difficult to initiate
open creative play, where neurotypical biases and habits don’t dominate
the creation of new ideas, and young neurodiverse collaborators are supported.
In my own research I have faced similar challenges in the design of
Accessible Digital Musical Instruments (ADMIs).
Very often the simplest instruments have proved to be the most
versatile and interesting.
The broad possibilities of simple musical tools have also been evident in
prior research into the design of digital musical instruments.
In 2010 the study shown here by Gurevich et al. found that a
single button instrument could support a diverse range of rhythmic styles,
physical stances and unanticipated experimental uses, and all in spite of
its extreme constraints.
Similarly, Zappi & McPhersons experiments with the DBox showed that users found
more creative potential and explored broader possibilities in the instrument
when it was configured with 1 rather than two dimensions of control over its sound.
The research presented here asks whether the relationship between constraints and
user creativity also applies in open play with young neurodiverse children,
in particular those who are often labelled as having complex needs.
If, as the studies suggest, constrained instruments require
lower cognitive bandwidth to operate, and instead allow a player to focus on other
aspects of performance, instruments such as these could prove useful for the
artistic applications we saw before, and for the development of
new accessible instruments.
A set of one-button sound cubes were built for the study. They featured a custom
analog force sensor, embedded speaker, amplifier, battery,
bluetooth module and a teensy board programmed using the Teensy Audio Library
with Faust, all housed in a 3D printed enclosure.
Three versions of the device were designed with different, but fixed sonic
behaviours, in order to accommodate different preferences in the way that
gesture, constrained instrument, and sound can relate to one another.
The first gave direct control over a single tone, the second was similar,
but offered control over a randomly generated tone row, and the third enabled
interactions with a non-linear feedback system.
The instruments were played by seven non-verbal young people at a
special school, in short, flexible solo sessions, up to 15 minutes long.
Using structural devices borrowed from inclusive theatre, the sessions were
broken up into three main sections – one for each instrument – with no verbal
instructions, and, where invited, adults could participate through
co-operative child led play. The sessions were recorded with a
compact video camera, and sensor data from the instruments was recorded via
bluetooth for later analysis.
Approaches to the analysis of these data were borrowed and adapted from the
constraint studies mentioned earlier, by Gurevich et al. and Zappi & McPherson.
The data is used to shed light only on very simple aspects of the players’
performances, as recorded by a neurotypical observer.
These aspects were physical gesture, and rhythmic style.
Video footage was used to categorise the gestures performed on the instruments.
Gestures were grouped and categorised as the analysis went on, defining a
‘new gesture’ as anything that would feel significantly different for the player in
performance. In order to do this, gestures were sketched as a means of
thinking through the physical processes involved, and in some cases, I also
attempted to re-enact them in order to gain a first-hand perspective.
These gestures were then plotted against the list of players, to explore where
there might be commonalities and differences in the gestures used
with the instruments.
The rhythmic fingerprint of each session was calculated and visualised in
Processing 3, using Gurevich et al.’s definitions of short, medium and
long durations for notes and silences: under a second for short durations,
medium between 1 and 3 seconds, and long durations above 3 seconds.
These durations were then mapped out on a timeline of notes and pauses for each
player, as well as being abstracted to a pie chart representing the distribution of
varying note/silence lengths within the entire session.
While actual footage from these sessions cannot be shown for ethical and
safeguarding reasons, what follows are some short excerpts from some of the
players’ sessions, and all of the individual gestures sketched
during the project…
Eighty-three gestures were recorded from the seven players’ sessions,
comprising 75 sounding gestures, and 8 non-sounding gestures.
These were further divided into squeezes, thumb presses,
hand/arm presses, strikes or taps, grabs or holds, swipes, mutes,
body presses, co-operative 2-player gestures,
stacking and arranging gestures. Only (16%) of these gestures were
not unique to one person, and only one gesture was shared by
more than two people.
A similar diversity is seen in the sensor data. Not only do the
visualised, rhythmic fingerprints of the players vary, but the consistency of
response to the different sound cubes does also. Some players were consistent
in their rhythmic approach, favouring a particular range of
durations, while others adjusted their rhythmic style depending on the
instrument being played.
Finally, it could be seen in the videos that many gestures evolved from
others during play, emerging as developments on a gestural theme.
These sometimes led to the discovery of hidden affordances, where features of the
instruments that weren’t originally intended to be musical were used as part
of an individual’s sonic-play.
These observations suggest a similar outcome to those in prior constraint
studies, albeit in a different context, that the constraints of the ADMIs
supported and facilitated diverse approaches to playing the sound
cubes, bringing about exploratory sonic play and the discovery of
hidden affordances.
These instruments were quickly and easily appropriated by the players.
And it is this last point that I’d like to expand upon
as this video comes to a close. The individuality that comes across
in each players’ first encounters with the instruments suggest possible avenues for
further collaboration in music or the design of bespoke ADMIs. For instance:
Player 1 was most engaged by cooperative press gestures, so future work might
explore a musical equivalent of Ahlquist’s Social Sensory Architectures,
where the most satisfying sensory results are only accessible through
cooperative play.
Player 5 played drones almost exclusively during their session and next steps might
include the design of drone-devices that create a subtle, immersive environment
that translates small body movements to changes in musical texture.
Player 7 seemed highly motivated by the vibrotactile potential of the instruments,
but later became more focused on the unpredictability of the white sound cube.
Future work could therefore explore strongly vibrating or resonant devices
with non-linear sound design.
As with any first impressions, there’s no guarantee that these ideas are sound,
but they do provide avenues for further work that are child led,
coming principally from their actions and responses.
Thanks to the clarity that a constrained ADMI provides, these responses are easier
to distinguish than they might otherwise be with a more complex instrument.
As such, I’d like to finish by proposing that highly constrained instruments
such as those discussed here needn’t be confined to academic studies, but could
prove to be a useful R&D tool for idea-generation in early-stage
collaborations with young neurodiverse people.
They provide low-cognitive and physical barriers to participation,
yet can encourage diverse individual responses.
I would be very interested to see how these can facilitate the co-production of
new performances and ADMIs in the future.
Although they can't be named here, I'd like to thank the seven young people
involved in this project for bringing their creativity and their ingenuity -
it wouldn't have been possible without them.
This work was done with ethical approval from the Royal Birmingham Conservatoire,
as well as complying with the participating school's
safeguarding regulations.