Distilled Post-Training Quantization and Quantization-Aware Training for CodeLlama-33B on Edge Devices

This report synthesises findings from 14 peer-reviewed papers addressing the following research question: How does the combination of knowledge distillation and post-training quantization (Distilled SPTQ) compare to other quantization-aware training (QAT) methods in terms of BLEU score preservation for. To help MLOps engineers decide which operator to use in which deployment scenario, this study aims to empirically assess the accuracy vs latency trade-off of white-box (training-based) and black-box operators (non-training-based) and their combinations in an Edge AI setup. We. 6 claims were extracted from source literature; 6 were independently verified against retrieved documents. An automated multi-reviewer quality assessment produced a score of 8.7/10. This report is a machine-generated literature synthesis and does not constitute original research.

Research goal: How does the combination of knowledge distillation and post-training quantization (Distilled SPTQ) compare to other quantization-aware training (QAT) methods in terms of BLEU score preservation for code generation tasks on CodeLlama-33B when deployed on edge devices?

Autonomous literature synthesis. Automated review score: 8.7/10. Full text and citation available at Assignee Research.