SimBench evaluation run (IEEE Access 2026): model-generated PyChrono scripts, multi-turn conversations, and J-LLM scores

This dataset is the frozen evaluation run accompanying the paper "SimBench: A Framework for Evaluating and Diagnosing LLM-Based Digital-Twin Generation for Multi-Physics Simulation" (IEEE Access, vol. 14, pp. 61784-61808, 2026; doi:10.1109/ACCESS.2026.3685519).

It holds the complete generated outputs and scores behind the published results. For 30+ student LLMs (S-LLMs) evaluated on the SimBench benchmark (34 Chrono/PyChrono physical systems x 3 turns = 102 tasks), it contains: the model-generated PyChrono scripts (i.e., virtual experiment scripts), their extracted/cleaned code, the rule-based LLM-as-a-judge (J-LLM) scores in three modes (documentation, reference, reference+documentation), and the multi-turn conversations in Alpaca format.

Files:
- output_llms.tar.gz (about 756 MB uncompressed): per-model, per-system generated scripts, the per-turn J-LLM score files, the per-system score CSVs, and the merged combined_evaluation_scores.csv.
- output_conversion.tar.gz (about 84 MB uncompressed): multi-turn conversation JSONs in Alpaca {instruction, input, output, history} format.
- SHA256SUMS: checksums for the two archives.

The benchmark code, the expert reference scripts, the API-documentation context, the rule-based judge, and the exact evaluation "contract" (judge model gpt-4o-mini with fixed sampling, the rubric, and a content hash of the task set) live in the source repository: https://github.com/uwsbel/SimBench . These archives are the large run outputs split out of that repository so the repo stays lean; scripts/fetch_published_data.sh restores them.

SHA256:
output_llms.tar.gz        29bc3c7f588d37c1ad839b92628d0ffcf7a30132944b56e4442fef4a31b7f591
output_conversion.tar.gz  7796e86dbae651570ab49e9598d6c8216dc34c72bf57c070eb72cf412222ba08