Bela-IREE: An Approach to Embedded Machine Learning for Real-Time Music Interaction

https://aimc2023.pubpub.org/pub/t2l10z49

Real-time artificial intelligence and machine learning processes are increasingly prevalent in creative technology practices, including in digital musical instrument (DMI) design and music interaction more broadly. However, achieving suitable performance in an embedded systems context, which is ideal for portable and self- contained instruments and interfaces, remains a challenge, with many different frameworks and approaches being explored simultaneously. In this work we explored the potential of combining the Intermediate Representation Execution Environment (IREE), part of the OpenXLA ecosystem, with the Bela embedded maker platform. We present a workflow and numerous tools for investigating the combination of Bela and IREE, including a virtualised container environment, an IREE runtime for Bela and an embedded model zoo. We report on the challenges of profiling embedded machine learning models, present initial benchmarking results, and conclude with future work towards a usable pipeline.