MARVEL - D4.1: Optimal audio-visual capturing, analysis and voice anonymisation – initial version

This report focuses on the initial version of the optimal audio-visual capturing, analysis and voice anonymisation. In the field of audio capturing, this document gives detailed information about the hardware developed and used in the scope of this project. Some consist of only the microphone and a simple connector, while others come with a processing unit that allows audio capturing via USB, streaming to a cloud via WiFi and even Edge AI processing. First versions of different data acquisition installations using the already available audio devices in different scenarios are presented, along with the first results of those experiments. This section concludes with information about plans for pre-processing for data analysis through the usage of Edge AI techniques. The second section of the document is about devAIce SDK, a modular technology optimised to work on cross-platforms and contains several AI models such as the Voice Activity Detection (VAD) tool as well as the feature extraction toolkit, openSMILE. It is used for audio analytics and sound event detection, acoustic scene classification and speech analysis which is a fundamental step to ensure privacy compliance and is designed to be implemented on high-end edge devices. Furthermore, an Android app used to record environmental acoustics and user annotations, SensMiner, is introduced.