Published June 1, 2023 | Version v1
Journal Open

XimSwap: Many-to-Many Face Swapping for TinyML

  • 1. E3DA Unit, Digital Society Center - Fondazione Bruno Kessler (FBK)

Description

The unprecedented development of deep learning approaches for video processing has caused growing privacy concerns. To ensure data analysis while maintaining privacy, it is essential to address how to protect individuals’ identities. One solution is to anonymize data at the source, avoiding the transmission or storage of information that could lead to identification. This study introduces XimSwap, a novel deep learning technique for real-time video anonymization, which can remove facial identification features directly on edge devices with minimal computational resources. Our approach offers a comprehensive solution that guarantees privacy by design. This novel method for implementing face-swapping ensures that the pose and expression of a target face remain unchanged and can be used on embedded devices with very limited computational resources. By incorporating style transfer layers into convolutional ones and optimizing the network’s operation, we achieved a reduction of over (98\% ) in the required operations and parameters compared to state-of-the-art architectures. Our approach also significantly reduces RAM usage, making it possible to implement the anonymization process on tiny edge devices, including microcontrollers, such as the STM32H743.

Notes

This work has been supported by the European Union’s Horizon 2020 research and innovation program under grant agreement no. 957337 (MARVEL project). This paper reflects only the authors’ views and the European Commission cannot be held responsible for any use which may be made of the information contained therein. Also, we acknowledge the support of the PNRR project FAIR - Future AI Research (PE00000013), under the NRRP MUR program funded by the NextGenerationEU.

Files

XimswapTECS2023_copyright.pdf

Files (10.4 MB)

Name Size Download all
md5:e77f98e7f8489e45305f321cba34d94b
10.4 MB Preview Download

Additional details

Identifiers

Funding

European Commission
MARVEL – Multimodal Extreme Scale Data Analytics for Smart Cities Environments 957337