State Dependency: Audiovisual interaction through brain states

Artistic installations using brain-computer interfaces (BCI) to interact with media in general, and sound in specific, have become increasingly numerous in the last years. Brain or mental states are commonly used to drive musical score or sound generation as well as visuals. Closed loop setups can emerge here which are comparable to the propositions of neurofeedback (NFB). The aim of our audiovisual installation State Dependency, driven by brain states and motor imagery, was to enable the participant to engage in unbound exploration of movement through sound and space unmediated by one's corpo-reality. With the aid of an adaptive feedback loop, perception is taken to the edge. We deployed a BCI to collect motor imagery, visual and cognitive neural activity to calculate approximate entropy (a second order measure of neural signal activity) which was in turn used to interact with the surround Immersive Lab installation. The use of entropy measures on motor imagery and various sensory modalities generates a highly accessible, reactive and immediate experience transcending common limitations of the BCI technology. State dependency goes beyond common practice of abstract routing between mental or brain with external audiovisual states. It provides new territory of unrestrained kinaesthetic and polymodal exploration in an immersive audiovisual environment.