CGCL-codes/Thorki: Thorki v1.0.0

🚀 New Release: Thorki v1.0.0

This release introduces Thorki, a federated learning system that decouples general and personalized knowledge across all model layers to achieve superior accuracy and efficiency.

✨ Key Features

• Dual-model design:
  • A small model for preserving client-specific personalized knowledge.
  • A large model for collaboratively learning general knowledge via federated training.
• Lightweight fusion network that adaptively combines outputs from both models during inference.
• Efficient personalization using a small adaptation set without compromising privacy.

🧪 Experimental Coverage

• Evaluated with three model architectures on four datasets under varying data heterogeneity.
• Includes comprehensive ablation studies validating each component of Thorki.

🔧 Implementation

• Supports CV and NLP tasks with pre-trained ViT and BERT backbones.
• Incorporates LoRA-based fine-tuning for parameter-efficient training.