Code and data for "Do Self-Improving Coding Agents Beat Textbook Baselines? Evidence from a Darwin Gödel Machine Port and Two Published Systems"

This record holds the code and data for the paper "Do Self-Improving Coding Agents Beat Textbook Baselines? Evidence from a Darwin Gödel Machine Port and Two Published Systems" by Tejasvi S. Tomar (https://doi.org/10.5281/zenodo.23720889). The same files are on GitHub at https://github.com/tejasvi/self-improving-agents-textbook-baselines (tag v4).

The archive contains the code of the study, the three pre-registrations and the change log. It also contains the per-task results of every evaluation in the paper, including the third pre-registered round, and every self-improvement loop, with each agent's edit. The re-evaluations of two published systems, the SWE-bench agents of the Darwin Gödel Machine and the agents discovered by ADAS, are included as well.

The command bash replay.sh regenerates the paper's numbers, tables and Figure 1 from the stored results. The command sha256sum -c --quiet SHA256SUMS then checks every file against the shipped checksums. The model calls themselves cannot be repeated without access to the same models.

LiveCodeBench problem statements are not redistributed. Each quoted statement is replaced by a placeholder that names the problem and its question id, so that the statement can be restored from LiveCodeBench.

The code is licensed under the Apache License 2.0. The data, the documentation and the paper source are licensed under CC BY 4.0. The file NOTICE lists the files derived from the Darwin Gödel Machine and from ADAS, which are licensed under the Apache License 2.0.