Virtual AI Jam: AI-Driven Virtual Musicians for Human-in-the-Loop Musical Improvisation

https://aimc2023.pubpub.org/pub/9064kmvw

This paper introduces an AI-driven and animated interactive musician system. Current AI-driven systems that accomplish interaction with a human musician often lack visual representation, or require building robots to play acoustic instruments. Our system provides an infrastructure for creating custom interactive musicians on a personal computer. Throughout the paper we introduce the system pipeline, our design process, feedback from semi-structured interviews with musicians, and how we addressed feedback to inform improvements and limitations for interaction. Findings suggest that many musicians desire interactive AI-driven improvisation systems that are explicitly controlled by the player as compared to free-form musical improvisation with autonomous virtual musicians. We share the interactive and animated AI musician system for public use and collective improvement, complete with an associated repository.