Neuromorphic Computing for On-Device LLM Inference: Why the Three-Layer Integration Gap Matters More Than the Algorithm Layer

Life of Research

Spiking neural networks (SNNs) on neuromorphic hardware are routinely proposed as the path to energy-efficient on-device
inference for language models. The framing usually treats this as a single technical question. It is not. There are three distinct
technical layers — SNN algorithms for language tasks, neuromorphic hardware platforms, and the integration of the two into
actual deployed systems — and each layer has a different state of progress, a different set of obstacles, and a different
research community. This paper makes three claims. First, conflating the layers obscures where the genuine gaps lie: SNN-LLM
algorithm research has accelerated substantially with SpikeGPT, SpikeLLM, Sorbet, and recent spike-driven LLM
constructions, and neuromorphic hardware has matured with Loihi 2, IBM NorthPole, SpiNNaker 2, and BrainChip Akida —
but actual deployed SNN-LLM systems running on neuromorphic chips at the edge remain almost nonexistent. The integration
layer is the bottleneck, not the algorithms. Second, the energy-efficiency case for neuromorphic-LLM deployment must be
made against the right baseline — INT4 or sub-4-bit quantized models running on modern mobile NPUs — not against
unquantized FP16 GPU baselines. The honest comparison narrows the advantage substantially and changes which workloads
are worth pursuing. Third, the most plausible near-term wins are not as drop-in replacements for mobile-NPU LLM inference
but in specific niches: always-on low-rate token processing, sensor-fused language tasks where input is already event-based,
and ultra-low-power deployment regimes where mobile NPUs do not operate. We propose a research agenda focused on the
integration gap and on rigorous baseline comparisons.