There is a newer version of the record available.

Published January 14, 2023 | Version v1
Preprint Open

One-Shot Federated Learning for Model Clustering and Learning in Heterogeneous Environments

  • 1. Carnegie Mellon University, Pittsburgh, PA
  • 2. Faculty of Technical Sciences, University of Novi Sad, Novi Sad
  • 3. Faculty of Sciences, University of Novi Sad, Novi Sad

Description

We propose a communication efficient approach for federated learn- ing in heterogeneous environments. The system heterogeneity is reflected in the presence of K different data distributions, with each user sampling data from only one of K distributions. The proposed approach requires only one communication round between the users and server, thus significantly reducing the communication cost. Moreover, the proposed method provides strong learning guarantees in heterogeneous environments, by achieving the optimal mean-squared error (MSE) rates in terms of the sample size, i.e., matching the MSE guarantees achieved by learning on all data points belonging to users with the same data distribution, provided that the number of data points per user is above a threshold that we explicitly characterize in terms of system parameters. Remarkably, this is achieved without requiring any knowledge of the underlying distributions, or even the true number of distributions K. Numerical experiments illustrate our findings and underline the performance of the proposed method.

Files

Armacki_etal_OneShotFL.pdf

Files (723.4 kB)

Name Size Download all
md5:e0326278498d6fc925c3c14d4504715f
723.4 kB Preview Download

Additional details

Funding

MARVEL – Multimodal Extreme Scale Data Analytics for Smart Cities Environments 957337
European Commission