FedShield-LLM: A Secure and Scalable Federated Fine-Tuned Large Language Model

FedShield-LLM is a novel framework that enables secure and efficient federated fine-tuning of Large Language Models (LLMs) across organizations while preserving data privacy. By combining pruning with Fully Homomorphic Encryption (FHE) for Low-Rank Adaptation (LoRA) parameters, FedShield-LLM allows encrypted computation on model updates, reducing the attack surface and mitigating inference attacks like membership inference and gradient inversion. Designed for cross-silo federated environments, the framework optimizes computational and communication efficiency, making it suitable for small and medium-sized organizations.

Paper: https://arxiv.org/abs/2506.05640
More relvant research: https://www.solidlab.network   

Acknowledgements This work is based upon the work supported by the National Center for Transportation Cybersecurity and Resiliency (TraCR) (a U.S. Department of Transportation National University Transportation Center) headquartered at Clemson University, Clemson, South Carolina, USA. Any opinions, findings, conclusions, and recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of TraCR, and the U.S. Government assumes no liability for the contents or use thereof.