Human-AI Musicking: A Framework for Designing AI for Music Co-creativity

https://aimc2023.pubpub.org/pub/zd46ltn3

In this paper, we present a framework for understanding human-AI musicking. This framework prompts a series of questions for reflecting on various aspects of the creative interrelationships between musicians and AI and thus can be used as a tool for designing creative AI systems for music. AI is increasingly being utilised in sonic arts and music performance, as well as digital musical instrument design. Existing works generally focus on the theoretical and technical considerations needed to design such systems. Our framework adds to this corpus by employing a bottom-up approach, as such it is built using an embodied and phenomenological perspective. With our framework, we put forward a tool that can be used to design, develop, and deploy creative AI in ways that are meaningful to musicians, from the perspective of musicking (doing music). Following a detailed introduction to the framework, we then introduce the four case studies that were used to refine and validate it, namely, a breathing guitar, a biosensing director AI, a folk-melody generator, and a real- time co-creative robotic score. Each of these is at different stages of development, ranging from ideation, through prototyping, into refinement, and finally, evaluation. Additionally, each design case also presents a distinct mode of interaction based on a continuum of human-AI interaction, which ranges from creation tool to co-creative agent. We then present reflection points based on our evaluation of using, challenging, and testing the framework with active projects. Our findings warrant future widespread application of this framework in the wild.