Published June 4, 2026
| Version v1
Software
Open
Artifact for MCIBench
Authors/Creators
Description
MCIBench is a multilingual code intelligence benchmark for evaluating large language models on problem-aligned code generation and directed code translation. It emphasizes same-problem multilingual evaluation: models are evaluated on the same programming problems across multiple mainstream programming languages, and accepted programs can be analyzed for functional correctness, runtime, and memory usage.
Benchmark Overview
| Item | Count |
|---|---|
| Problems | 1,489 |
| Languages | 8 |
| Generation qid-language slots | 11,912 |
| Directed translation language pairs | 56 |
| Translation task problems | 100 |
| Original reference-complete problems across all 8 languages | 332 |
| Augmented complete problems using accepted experimental solutions | 1,466 |
The 8 languages are C, C++, C#, Java, JavaScript, Python3, Golang, and Rust.
Files
MCIBench.zip
Files
(65.7 MB)
| Name | Size | Download all |
|---|---|---|
|
md5:8305f91651748b34b779b70fb993fd53
|
65.7 MB | Preview Download |