Video/Audio Open Access

Neural Touch-Screen Ensemble Performance 2017-07-03

Martin, Charles Patrick

A studio performance of an RNN-controlled Touch Screen Ensemble from 2017-07-03 at the University of Oslo.

In this performance, a touch-screen musician improvises with a computer-controlled ensemble of three artificial performers. A recurrent neural network tracks the touch gestures of the human performer and predicts musically appropriate gestural responses for the three artificial musicians. The performances on the three 'AI' iPads are then constructed from matching snippets of previous human recordings. A plot of the whole ensemble's touch gestures are shown on the projected screen.

This performance uses Metatone Classifier (https://doi.org/10.5281/zenodo.51712) to track touch gestures and Gesture-RNN (https://github.com/cpmpercussion/gesture-rnn) to predict gestural states for the ensemble. The touch-screen app used in this performance was PhaseRings (https://doi.org/10.5281/zenodo.50860).

This work is supported by The Research Council of Norway as a part of the Engineering Predictability with Embodied Cognition (EPEC) project, under grant agreement 240862.
Files (248.5 MB)
Name Size
neural-touch-screen-session-2017-07-03.mp4
md5:ed4904bb1d053d17cc9c953a3593962e 		248.5 MB Download

  • Martin, C., & Swift, B. (2016). MetatoneClassifier: Research Prototype. Zenodo. http://doi.org/10.5281/zenodo.51712

  • Martin, C. (2016). PhaseRings v1.2.0. Zenodo. http://doi.org/10.5281/zenodo.50860

  • Martin, C., Gardner, H., & Swift, B. (2015). Tracking ensemble performance on touch-screens with gesture classification and transition matrices. In Proceedings of the International Conference on New Interfaces for Musical Expression, NIME '15, pages 359–364. http://www. nime.org/proceedings/2015/nime2015_242.pdf

52
10
views
downloads
All versions This version
Views 5251
Downloads 1010
Data volume 2.5 GB2.5 GB
Unique views 5049
Unique downloads 1010
More info on how stats are collected.
Indexed in
Publication date:
July 19, 2017
DOI:
10.5281/zenodo.831910

Zenodo DOI Badge

DOI 

10.5281/zenodo.831910

Markdown 

[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.831910.svg)](https://doi.org/10.5281/zenodo.831910)

reStructedText 

.. image:: https://zenodo.org/badge/DOI/10.5281/zenodo.831910.svg
   :target: https://doi.org/10.5281/zenodo.831910

HTML 

<a href="https://doi.org/10.5281/zenodo.831910"><img src="https://zenodo.org/badge/DOI/10.5281/zenodo.831910.svg" alt="DOI"></a>

Image URL 

https://zenodo.org/badge/DOI/10.5281/zenodo.831910.svg

Target URL 

https://doi.org/10.5281/zenodo.831910

Keyword(s):
performance, music, touch-screen, improvisation, recurrent neural network
License (for files):
Creative Commons Attribution 4.0 International

Versions

Version 1 10.5281/zenodo.831910 Jul 19, 2017
Cite all versions? You can cite all versions by using the DOI 10.5281/zenodo.831909. This DOI represents all versions, and will always resolve to the latest one. Read more.

Share

Cite as

Export