Shackled to the Machine — Breaking the DeepSeek-R1-8B Public Record by +29% on Four Raspberry Pi 5 | Hellomatik

We evaluate CPU-only distributed inference of DeepSeek-R1-Distill-Llama-8B (dense 8B, Q40) on a four-node Raspberry Pi 5 (16 GB) cluster running distributed-llama.

A clean-room measurement at the optimal thread count sustains 8.32 tok/s short-context decode (n=20, 95% CI [8.30, 8.34]), bit-exact — to our knowledge the best published figure for this model on Raspberry Pi 5, 29% above the prior four-node record (6.43 tok/s) and 37% above the Seeed Studio configuration (6.06 tok/s).

We attribute the gain by layer: the production binary is byte-identical to clean upstream, so the source/kernel changes contribute approximately zero to dense decode (they add 86% to prefill and 15.2% to a 30B Mixture-of-Experts model in the companion report).

Dense decode is bound by memory bandwidth; the gains live in the hardware and operating-system layers, not the application kernels.

Using distributed-llama's per-token timing we show decode falls roughly linearly with context length — t(L) = 117 + 0.0174·L ms/token, R² = 0.999, prompt-independent — to about 5.0 tok/s at a 5,000-token context, so the 8.32 figure is the short-context peak.

The four-thread optimum is context-independent (≈13% over three threads at every length).

We additionally describe a bit-exact, adaptive prompt-lookup speculative decoder and a controlled dense-vs-MoE contrast on identical silicon.

All figures are bit-exact (SHA-256 of the generated token-id stream at temperature 0, seed 42); no overclocking, quantisation change, or model substitution.

Raw per-token timing data, benchmark harness and LaTeX sources are included.