Hello everyone I am from Georgia Tech.
I am here to present this paper titled
Mechatronics-Driven Musical Expressivity
for Robotic Percussionists.
Robotic musicianship can be divided
into two sub-research directions.
machine musicianship focused on
developing algorithms and
cognitive models of musicianships.
And music mechatronics studies
the construction of physical systems that
generate sound through mechanical means.
In music mechatronics,
Researchers utilize various actuators,
sensors and control techniques,
with the consideration of acoustic designs
to create human-inspired robotic musicians.
There are many currently existed robotic percussionists,
Such as Cog robot from MIT,
the Humanoid drummer by Mitsuo Kawato,
MahaDeviBot,
Haile from Georgia Tech,
and even a Robotic Drumming Prosthetic
for an amputee drummer.
but these platforms cannot really support
human-level dynamic range
or millisecond micro-timings.
little progress has been made
to improve the acoustic expressivity
of the robotic musicians through
the advancements in Musical Mechatronics.
In this paper, we enhanced
musical expressivity on Shimon,
a robotic marimba-player.
Previously, Shimon had a
solenoid-based striking system,
which limited dynamic range and speed.
Our goal was to improve Shimon's
expressivity with a better
dynamic range and speed,
by implementing a brushless
DC motor striking system.
We hypothesize that
by implementing bldc motors,
Shimon will gain wider
dynamic range and speed than solenoids,
and reproduce the human-level expressivity.
Maxon EC-60 Flat Motor,
and Maxon EPOS4 50/8 Controllers
are used to implement in the
new striking system.
A Beckhoff Industrial PC is used to execute
motion commands through PLC system
A model of the mallet is built to analyze
the kinematics and dynamics of the strokes.
With the model, we are able interpolate
the output torque of the motor to
control dynamic range.
We are also able to find relationship
between torque and acceleration,
and then maintain the constant stroke time
for different stroke dynamics.
In this case, we are able to achieve
micro timing controls in wide dynamic range.
Musicians just need to input their MIDI note
from a DAW, and a MAX/MSP patch will
transfer the midi signals to UDP messages,
and then send to IPC to trigger the motion.
The homing of encoder value is done at
the stage of hardware initialization.
Once a command is received,
the mallet will be routed to the node by
the linear actuators,
and raise to the initial hit position.
Once the initial hit position is reached,
the mallet will be manipulated by the motor
and hit the marimba key.
While a loop is checking whether
velocity rating is below a threshold.
When the threshold is reached,
meaning that the mallet has hit the key,
the motor will raise the mallet back up to
its default position
and wait for the next command to come.
We performed both dynamic range
and speed tests on the solenoids
and Brushless DC motors.
For dynamic range,
both solenoids and BLDC motors were
programmed to play single C2 note
at a speed of 60 bpm on the marimba
and MIDI velocity is at ascending of
step 1 from 1to 127 for every 6 strokes.
For speed, Both of these actuators were
programmed to play single C2 note for
3 seconds, with highest system defined speed.
We also invited 21 participants to
join our subjective test.
They were asked to participate
in a 30- minute listening test
to compare expressivity of the excerpts
among solenoids, BLDC motors, and 4 other
human marimba players.
Now, I invite everyone to do a listening test,
and see if you can differentiate.
The subjective tests begin with an
introduction to musical expressivity,
aimed at training subjects to identify
appropriate dynamic range and micro timing,
as well as their effects on perceived expressivity.
After training,
the subjects listen to the excerpts and answer
Which sample is more musically expressive
The results show insignificance between
BLDC motors and humans.
A p value of 0.19 indicates that statistically,
the subjects believe that
The BLDC motor can play equally expressive
excerpts to humans.
While the comparisons between both
human and solenoid,
BLDC motor and solenoids,
are showing significance.
It proves that the solenoid is less expressive
than either human or BLDC motors.
Here is the result of the objective test.
We can clearly see that the BLDC motor
obtained a larger dynamic range than
the solenoids, and also a much more stable
and linear step increase of sound intensity
along with the commanded MIDI velocities.
Speedwise,
at highest stable hits,
BLDC motor reached a speed of 32.9 Hz,
far more better than the 8.3 Hz of the solenoids.
The implementation of BLDC motors achieved
goal of wider dynamic range and higher speed.
The expressivity performed by the new system
is also proved to be similar to human players.
To end this,
let take a look at a stage performance of
the new system,
with some MV scenes filmed with the new Shimon.
Thank you very much, and enjoy.