Knowledge Distillation from Large to Small Language Models for Efficient Code Generation

This report synthesises findings from 12 peer-reviewed papers addressing the following research question: To what extent can knowledge distillation from large language models improve the inference efficiency of small language models in code generation tasks, as evaluated by latency and pass@k metrics on. In the last few years, the deep learning (DL) computing paradigm has been deemed the Gold Standard in the machine learning (ML) community. Moreover, it has gradually become the most widely used computational approach in the field of ML, thus achieving outstanding results on. 8 claims were extracted from source literature; 8 were independently verified against retrieved documents. An automated multi-reviewer quality assessment produced a score of 8.7/10. This report is a machine-generated literature synthesis and does not constitute original research.

Research goal: To what extent can knowledge distillation from large language models improve the inference efficiency of small language models in code generation tasks, as evaluated by latency and pass@k metrics on HumanEval and DS-1000 benchmarks?

Autonomous literature synthesis. Automated review score: 8.7/10. Full text and citation available at Assignee Research.