Quantization Impact on HumanEval Pass@1 Scores in Code Generation Models

This report synthesises findings from 1 peer-reviewed paper addressing the following research question: How does 4-bit versus 8-bit quantization impact the HumanEval pass@1 scores of code generation models when evaluated on different programming languages. Democratization of AI is an important topic within the broader topic of the digital divide. This issue is relevant to LLMs, which are becoming popular as AI co-pilots but suffer from a lack of accessibility due to high computational demand. 11 claims were extracted from source literature; 11 were independently verified against retrieved documents. An automated multi-reviewer quality assessment produced a score of 8.0/10. This report is a machine-generated literature synthesis and does not constitute original research.

Research goal: How does 4-bit versus 8-bit quantization impact the HumanEval pass@1 scores of code generation models when evaluated on different programming languages?

Autonomous literature synthesis. Automated review score: 8.0/10. Full text and citation available at Assignee Research.