Hello, my name is Pelle,
and I'm a masters student at Aalborg University, Copenhagen.
This presentation will be about this instrument, called the daïs,
which I've developed in collaboration with Stefania Serafin and Dan Overholt.
First I'll talk about the of design instrument
then I'll go through development process,
then I'll talk about sound synthesis and parameter mapping,
and finally, I'll cover our evaluation of the instrument.
The basic idea for the instrument was to have a suspended disk
that one could move in 3D using your hand.
The position and rotation of the disk
would give us six values that could be mapped
to a sound synthesis algorithm.
This would allow us to control many parameters
at the same time using only one hand.
Now, when a performer plays an acoustical instrument,
we usually think of audition as the primary modality.
However, vision and touch usually also play a role in the interaction.
The player receives sensory cues from the different modalities
which are then combined to make a multimodal percept of what is going on,
which the player uses to guide their actions.
This forms a tight feedback loop
between the player and the instrument
which is important for expressive playing.
For this reason, we wanted to add active haptic feedback to the daïs.
Since haptic feedback is something that is often overlooked
in electronic music instruments.
For development we focused on an iterative methodology.
The goal of the first iteration was to get a proof of concept
as fast as possible.
The physical construction was made from cardboard
and consisted of a disk suspended by rubber bands.
Position sensing was achieved using magnets and Hall effect sensors.
These were read by a Teensy,
which would send MIDI messages to a laptop
running some Faust code to generate sound.
Small vibrotactile actuactors were added to the disk for haptic feedback.
Additionally, the rubber bands suspending the disk
provided passive haptic feedback by pushing back at you
whenever you pushed the disk down.
From the first iteration we learned that
Hall effects sensors were not a good choice,
but otherwise the interface seemed promising.
For the second iteration we went with a similar design,
but this time built from laser cut fiber board.
Instead of the Hall effect sensors, we now used optical proximity sensors.
To capture the tilt of the disk, three sensors were needed,
which meant we had to develop a custom PCB for reading the sensors
and holding them in known positions.
At this point we switched from the Teensy/laptop
to a Bela to make the instrument self-contained.
Also, the small vibrotactile transducers were
replaced with bigger ones.
Moving on to the third iteration,
we wanted to give the instrument a more finished look,
and moved on to this barrel-like design.
While evaluating the second iteration iteration,
we found that the three-sensor configuration
did not work as well as we wanted.
Instead we used a single sensor to measure how far the disk is pressed down
and an IMU to capture the tilt of the disk.
The rubber bands we're replaced with an elastic string,
which allowed for more flexibility in how the disk is suspended.
The sound synthesis algorithm we use is a
physical model of a bowed string
using finite difference time domain methods.
The string is modeled by the 1D wave equation
with frequency dependent damping.
The bow is modeled by a friction function
that depends on the relative velocity between the bow and the string.
The parameters of interest in this model is the wave speed,
which controls the pitch of the string,
the bowing force, which is how hard the bow is pressed onto the string,
and lastly, the speed of the bow.
The wave speed is controlled by an
external MIDI keyboard that plugs into the Bela.
The bowing force is mapped to the depression of the disk.
So when I press the disk down
it corresponds to putting the bow on the string.
The bowing speed is controlled by the tilt of the disk,
so when I tilt the disk towards myself
I start the bow moving
If I tilt the disk away from myself
I stop the bow moving.
To produce sound I will first press down the disk
to place the bow on the string.
Then I'll tilt it towards myself to start moving the bow,
which makes the string oscillate.
You can't really see it,
but the disk is actually vibrating right now
due to the vibrotactile actuators.
If I stop moving the bow,
the string will stop oscillating
because the bow is still in contact with the string.
If I lift the bow while still moving it,
the string is allowed to ring out,
because the bow is not touching the string.
The wave speed is also mapped to the
side-to-side tilt of the disk,
which can be used to create vibrato.
Additionally, we have placed a contact microphone on the side of the daïs,
which can also be used to excite the string.
At first, we wanted to have the microphone on the disk,
but that caused issues in combination with the haptic feedback,
so we moved it to the side instead.
For evaluation we did a simple interview-style test on six participants.
The goal was to see what they thought of the mapping,
how it felt for them to play the daïs,
and to get ideas for future development.
First the test subjects were asked questions
and then allowed to play the instrument,
without us explaining how the instrument worked.
Afterwards, we would explain the mapping,
let them play the instrument again,
and then ask some follow-up questions.
The test subjects we're overall impressed
with the look and feel of the instrument
and most found the test to be an enjoyable experience.
However, the mapping is unintuitive.
The design of the instrument does not give
affordance to the chosen mapping,
and no subjects were able to figure it out
by themselves.
Actually, many participants thought the instrument looked like a drum,
and some expected a drum-like interaction
and a percussive sound.
No participants expected the bow-string sound. That we actually implemented.
One participant discovered a playing technique
where they tapped the contact mic an then pressed down the disk
to produce a sustained tone with a sharp attack.
A couple of other participants we're able
to play the instrument somewhat musically,
but overall, the short duration of the test
was not enough to learn the instrument.
Only one subject commented on the haptic feedback by themselves,
and thought it was a nice feature.
Other subjects, when inquired about it,
thought the same but had not been actively thinking about it.
This suggests to us that the haptic feedback
extends the instrument in a natural way
that reinforces the interaction.
In the future we would like to try different
mappings and synthesis algorithms.
The evaluation suggested
hat a drum-like interaction would be more suitable,
so that might be something to try.
There are still a few issues with the hardware:
the construction could be a bit more sturdy
and there are problems with noise level
of the proximity sensor at far distances.
If you're interested, all the code and the CAD drawings
for the daïs are available online on GitHub,
with instructions on how to build the instrument.
That was everything from me, thank you for watching!