QES Characterization: Data-Path, Cache Locality and Energy in Single-Node Trillion-Parameter MoE Inference

QES Characterization: Data-Path, Cache Locality and Energy in Single-Node Trillion-Parameter MoE Inference is a complementary systems-characterization paper to the QES v2.7 white paper.

The study examines where generated-token time is spent when executing Kimi K2.5 on a single 128 GB AMD Ryzen AI MAX+ 395 system using storage-backed bounded expert residency. It measures generated-only expert traffic, routing locality, cache effectiveness, sustained decode behaviour and package energy.

The measured 128-token workload contains 61,440 generated expert requests. A bounded locality cache records 4,734 generated-only cache hits (7.705%), reducing expert-store traffic from a 7.126 GiB/token no-cache logical requirement to 6.577 GiB/token and avoiding 70.285 GiB of expert traffic over the run. Mean expert-delivery wait falls by approximately 5%, while repeated cache-enabled runs remain close to 0.438 tokens/s compared with the frozen 0.432091702 tokens/s baseline. A package-power repeat measures 113.47 J/generated token versus 117.50 J/generated token for the no-cache reference.

The results support storage-backed bounded residency as a feasible engineering route for very large sparse Mixture-of-Experts models where full model residency is unavailable and latency tolerance exists. The work does not claim interactive serving parity, universal model compatibility or production readiness.

This publication intentionally reports measured behaviour and architecture-level findings without disclosing the private QES implementation, source code, expert-store construction, scheduling logic, residency-control mechanisms or model-graph integration.

Parent publication: Nigel Hutchinson / NH Applied, QES: Bounded-Memory Execution of Large Mixture-of-Experts Models on a Single 128 GB System — From Qwen3-235B to Kimi K2.5, v2.7. DOI: 10.5281/zenodo.22730031