Planned intervention: On Wednesday April 3rd 05:30 UTC Zenodo will be unavailable for up to 2-10 minutes to perform a storage cluster upgrade.

There is a newer version of the record available.

Published January 10, 2022 | Version v1
Project deliverable Open

MARVEL - D4.1: Optimal audio-visual capturing, analysis and voice anonymisation – initial version

Creators

  • 1. IFAG

Description

This report focuses on the initial version of the optimal audio-visual capturing, analysis and voice anonymisation. In the field of audio capturing, this document gives detailed information about the hardware developed and used in the scope of this project. Some consist of only the microphone and a simple connector, while others come with a processing unit that allows audio capturing via USB, streaming to a cloud via WiFi and even Edge AI processing. First versions of different data acquisition installations using the already available audio devices in different scenarios are presented, along with the first results of those experiments. This section concludes with information about plans for pre-processing for data analysis through the usage of Edge AI techniques. The second section of the document is about devAIce SDK, a modular technology optimised to work on cross-platforms and contains several AI models such as the Voice Activity Detection (VAD) tool as well as the feature extraction toolkit, openSMILE. It is used for audio analytics and sound event detection, acoustic scene classification and speech analysis which is a fundamental step to ensure privacy compliance and is designed to be implemented on high-end edge devices. Furthermore, an Android app used to record environmental acoustics and user annotations, SensMiner, is introduced.

Files

MARVEL-d4.1.pdf

Files (2.1 MB)

Name Size Download all
md5:cc3dd6c9bebb4dc8dd228e4277baf082
2.1 MB Preview Download

Additional details

Funding

MARVEL – Multimodal Extreme Scale Data Analytics for Smart Cities Environments 957337
European Commission