PhiNet-GAN: Bringing real-time face swapping to embedded devices

Recent years have seen an unprecedented development of deep learning-based techniques for processing live video from CCTV cameras, causing growing privacy concerns. A possible solution is to ensure that a subject's personal information never leaves the device in which it was collected, thus implementing a Privacy-by-Design (PbD) approach. In live video processing tasks, PbD can be guaranteed through anonymisation techniques, such as face-swapping, performed directly on the end device. This paper, therefore, presents PhiNet-GAN, an extension of the PhiNet family of embedded neural networks applied to generative networks. PhiNet-GAN targets resource-constrained platforms based on low-power microcontrollers. An example is the Kendryte K210, a RISC V dual-core processing unit working at 400MHz on which we tested our network. Overall we achieved a power consumption of less than 300mW, working at more than 15fps with an FID score lower than 150.