Published June 23, 2026 | Version v1

Calliphony: A Calligraphy-Driven Interface for Real-Time Generative Music Performance

Description

While music generative models have recently gained significant attention, how they can be effectively integrated into live music performances still requires further exploration. This paper presents Calliphony, a calligraphy-driven interface for real-time generative music performance. Specifically, we build a low-latency pipeline that captures brush motion with an attachable sensor and maps it to control signals for real-time symbolic music generation. Using a generative model, the system produces multi-track MIDI in performance settings, while brush-derived control signals constrain event timing and activate additional musical layers.The generated melody is then extended with real-time harmony and additional voices, and finally rendered through a DAW for live staging.Calliphony contributes: (1) a performance-oriented prototype that uses calligraphic motion as an external control layer for a real-time symbolic music generation model, controlling note density, pitch constraints, and accompaniment-layer activation; and (2) a cross-modal performance scenario that extends calligraphy beyond a primarily visual practice into an audiovisual, AI-assisted setting.

Files

nime2026_32.pdf

Files (10.8 MB)

Name Size
md5:60fa04765ab203743fdc50a0155b2ca3
10.8 MB Preview Download