How do alignment techniques (e.g., RLHF, DPO) affect the trade-off between MATH accuracy and inference efficie

In this report, we introduce Qwen2.5, a comprehensive series of large language models (LLMs) designed to meet diverse needs. Compared to previous iterations, Qwen 2.5 has been significantly improved during both the pre-training and post-training stages. In terms of pre-training, we have scaled the high-quality pre-training datasets from the previous 7 trillion tokens to 18 trillion tokens. This provides a strong foundation for common sense, expert knowledge, and reasoning capabilities. In terms of post-training, we implement intricate supervised finetuning with over 1 million samples, as well

Research goal: How do alignment techniques (e.g., RLHF, DPO) affect the trade-off between MATH accuracy and inference efficiency (e.g., tokens/sec) in Claude and Gemini models?

Autonomous synthesis report generated by SOVEREIGN Research Kernel. Tribunal consensus score: 9.0/10.