Extending Deep Rhythm for Tempo and Genre Estimation Using Complex Convolutions, Multitask Learning and Multi-input Network

Tempo and genre are two inter-leaved aspects of music, genres are often associated to rhythm patterns which are played in specific tempo ranges. In this paper, we focus on the recent Deep Rhythm system based on a harmonic representation of rhythm used as an input toa  convolutional  neural  network.  To  consider  the  relationships  between frequency  bands,  we  process  complex-valued  inputs  through  complex-convolutions. We also study the joint estimation of tempo/genre using a multitask learning approach. Finally, we study the addition of a second input branch to the system based on a VGG-like architecture applied toa mel-spectrogram input. This multi-input approach allows to improve the performances for tempo and genre estimation.