Stacco: Exploring the Embodied Perception of Latent Representations in Neural Synthesis

The application of neural audio synthesis methods for sound generation has grown significantly in recent years. Among such systems, streaming autoencoders such as RAVE are particularly suitable for instrument design, as they map audio to and from control signals in an abstract latent space with acceptable latency. Despite the uptake of autoencoders in NIME design, little research has been done to characterize the latent spaces of audio models, and to investigate their affordances in practical musical scenarios. In this paper we present Stacco, an instrument specifically designed for the intuitive control of neural audio synthesis latent parameters through the displacement of magnetic objects on a wooden board with four magnetic attractors. We then examine models trained on the same data with different seeds, we explore strategies for more consistent mappings from audio to latent space, and propose a method for stitching the latent space of one model to another. Finally, in a user study, we investigate whether and how these techniques are perceived through embodied practice with Stacco.