Published November 12, 2026 | Version 1.0.0

LeanRoute-216: dataset for "The Shape of Mathematical Creativity: Measuring Mathematical Exploration in Formal Proof Generation"

  • 1. Clemson University

Description

LeanRoute-216: 216 kernel-verified Lean proofs for 36 fixed propositions (36 human-reviewed originals plus 180 generated variants), organized as parent-to-candidate edges with benchmark, machine and human route labels, and 955 LLM-judge route judgments from six models (930 held-out, 25 development). Provided as the original workbook and as two CSV exports (proofs and model judgments).

Dataset accompanying the paper The Shape of Mathematical Creativity: Measuring Mathematical Exploration in Formal Proof Generation, 6th Workshop on Mathematical Reasoning and AI (MATH-AI) at NeurIPS 2026 (https://mathai-2026.github.io/).

Paper abstract. Proof-generating systems return many candidates for the same theorem. Lean's kernel determines whether each candidate is correct, but not whether the candidates contain different mathematical reasoning. We study one measurable component of mathematical creativity: whether repeated successful generations explore distinct proof ideas rather than new formal expressions of the same idea. We analyze LeanRoute-216, which began with 36 kernel-verified and human-reviewed Lean proofs for 36 fixed propositions. Five additional proofs were then generated for each proposition with the goal of obtaining diverse proofs, producing 216 kernel-verified proofs in total. A graph-derived route-level review identified 61 diverse routes among the 216 distinct proof strings. This observation motivated a closer study of the distinction between formal variation and mathematical route diversity. Across the benchmark's 180 parent-candidate pairs, 25 were labeled as route changes and 155 as same-route comparisons. All pairs of candidate proofs were human-evaluated in a blinded second pass, with 95.6% agreement with the original route-change labels (κ = 0.793). Five theorem panels were used to refine the rubric and judging prompts, while the remaining 31 panels were held out for evaluation. On these 31 panels, LLM judges recognize route changes more reliably than lexical baselines, but remain substantially weaker at identifying the kind of change. These results support route diversity as an evaluation target separate from correctness and show how route-level auditing can inform proof search and the curation of synthetic reasoning data.

Known limitations.

  • Route labels are human judgments on a small panel (36 propositions); five panels were used for rubric development.

License: Creative Commons Attribution 4.0 International (https://creativecommons.org/licenses/by/4.0/).

Files

croissant.json

Files (126.8 kB)

Name Size
md5:8c70f6f30481e7c4ac11074b20ad8576
1.5 kB Download
md5:812dfbcd845d67bdab5052eec0a11ef3
13.4 kB Preview Download
md5:783c3625a8fcbf73561fe15923b07e42
106.2 kB Preview Download
md5:7f3dcd5fab39c80b2d0fc53823b55bdf
379 Bytes Download
md5:b471931febd3901a82d47068d3932e00
5.4 kB Preview Download

Additional details

Related works

Is identical to
Dataset: https://huggingface.co/datasets/ai4he/leanroute-216 (URL)
Is supplement to
Conference paper: https://mathai-2026.github.io/ (URL)