Cybernesis: An Improvisational Performance System Exploring Gesture Control of Hardware Synthesizers

Cybernesis is a performance piece that explores the interaction between human gesture, machine learning, and real-time sound synthesis. Using a Leap Motion controller, the performer’s hand movements are captured and analyzed by custom software. This gestural data trains a multilayer perceptron (MLP), a form of neural network, which in turn predicts and influences the internal state of a complex hardware sound synthesis system. The core of the performance lies in the real-time exploration of nonlinear mapping functions through linear regression, navigated spatially through the performer’s listening and intuitive hand movements. This creates a dynamic feedback loop where the performer and the model co-create the sonic output, positioning the learning algorithm not merely as a tool, but as an active participant in the improvisational process mediated by the performer’s embodied interaction.