ExpertFlow: Efficient Mixture-of-Experts Inference via Predictive Expert Caching

Sparse Mixture-of-Experts (MoE) models can outperform dense large language models at similar computation by activating only a small set of experts per token. However, stacking many expert modules introduces substantial parameter memory, which makes MoE models difficult to deploy in memory-constrained environments such as single-GPU devices. Offloading alleviates this issue by storing inactive experts in CPU memory and loading them on demand, but existing methods remain limited: static caches disregard input-dependent routing, and methods that train separate models to predict expert usage ahead

Research goal: Can SMoES routing be combined with activation-aware quantization (e.g., AWQ, GPTQ) to improve tokens-per-second throughput on A100/H100 GPUs without degrading ChartQA and DocVQA accuracy below dense model baselines?

Autonomous synthesis report generated by SOVEREIGN Research Kernel. Tribunal consensus score: 7.5/10.