Cold Neuron Pruning and Code Generation Accuracy in PowerInfer's Sparse Activation Pipeline

This report synthesises findings from 4 peer-reviewed papers addressing the following research question: What is the relationship between activation sparsity ratios and code generation accuracy degradation in state-spaces/lm-eval-harness when pruning to cold neurons only in PowerInfer's pipeline. This paper introduces PowerInfer, a high-speed Large Language Model (LLM) inference engine on a personal computer (PC) equipped with a single consumer-grade GPU. The key principle underlying the design of PowerInfer is exploiting the high locality inherent in LLM inference. 7 claims were extracted from source literature; 7 were independently verified against retrieved documents. An automated multi-reviewer quality assessment produced a score of 9.0/10. This report is a machine-generated literature synthesis and does not constitute original research.

Research goal: What is the relationship between activation sparsity ratios and code generation accuracy degradation in state-spaces/lm-eval-harness when pruning to cold neurons only in PowerInfer's pipeline?

Autonomous literature synthesis. Automated review score: 9.0/10. Full text and citation available at Assignee Research.