YouTube Mirror: An Interactive Audiovisual Installation based on Cross-Modal Generative Modeling

https://aimc2023.pubpub.org/pub/m5cbcx8i

YouTube Mirror is an interactive, audiovisual AI installation that generates images and sounds in response to the images of the audience captured through a camera in real time in the form of an interactive mirror. YouTube Mirror uses a cross-modal generative machine model that was trained in an unsupervised way to learn the association between images and sounds obtained from the YouTube videos I watched. Since the machine represents the world only based on the learned audio-visual relationship, the audience can see themselves through the machine-generated images and sounds. YouTube Mirror is an artistic attempt to simulate my unconscious, implicit understanding of audio-visual relationships that can be found in the videos. As an empirical case study, this project also aims to investigate the possibility of the use of cross-modal generative modeling in the production of interactive audiovisual installations. This paper describes the background of the project and the design, implementation, and exhibition of the YouTube Mirror installation, followed by the discussion and future work.