Automated Test Suite Robustness for Mistral-Large-2 Code on LiveCodeBench

This report synthesises findings from 5 peer-reviewed papers addressing the following research question: What is the robustness of automated test suite evaluations for code generated by Mistral-Large-2 on MBPP when benchmarked against human evaluations using Cohen's kappa for inter-rater agreement. Large Language Models (LLMs) applied to code-related applications have emerged as a prominent field, attracting significant interest from both academia and industry. However, as new and improved LLMs are developed, existing evaluation benchmarks (e.g., HumanEval, MBPP) are no. 7 claims were extracted from source literature; 7 were independently verified against retrieved documents. An automated multi-reviewer quality assessment produced a score of 8.7/10. This report is a machine-generated literature synthesis and does not constitute original research.

Research goal: What is the robustness of automated test suite evaluations for code generated by Mistral-Large-2 on MBPP when benchmarked against human evaluations using Cohen's kappa for inter-rater agreement?

Autonomous literature synthesis. Automated review score: 8.7/10. Full text and citation available at Assignee Research.