Replication package for "Fully Autonomous Programming using Iterative Multi-Agent Debugging with Large Language Models"

This repository contains the replication package for the paper Fully Autonomous Programming using Iterative Multi-Agent Debugging with Large Language Models, Anastasiia Grishina, Vadim Liventsev, Aki Härmä, and Leon Moonen, under review for ACM Transactions on Evolutionary Learning and Optimization.

The replication package is archived on Zenodo with DOI: 10.5281/zenodo.13754705. The source code is distributed under the MIT license, the data is distributed under the CC BY 4.0 license.

Organization

The repository is organized as follows:

• Archived source code in the src folder, with a dedicated README.

• Analysis of the results in the analysis folder, with a dedicated README.

• Archive of SEIDR-generated solutions as a git bundle: seidr-solutions.bundle.

  • These solutions can be inspected by cloning the bundle using git clone seidr-solutions.bundle which will create a folder seidr-solutions with a dedicated README in the master branch.
  • The other branches contain the code that was generated in specific experiments/configurations. List which experiments are available using git branch -r, select one using git checkout <branch>, and look at the iteratively synthesized solution for a problem using git log -p -- {problem}.{cpp/py}. Alternatively, these can be inspected on GitHub.

Citation

If you build on this data or code, please cite this work by referring to the paper:

@article{grishina2025:fully,
   title = {Fully Autonomous Programming using Iterative Multi-Agent Debugging with Large Language Models},
   author = {Anastasiia Grishina and Vadim Liventsev and Aki Härmä and Leon Moonen},
   journal = {Transactions on Evolutionary Learning and Optimization},
   year = {2025},
   publisher = {ACM}
   note = {Under review}
}

External References

• The SEIDR source code is maintained on GitHub.
• The SEIDR-generated solutions for PSB2 are versioned on GitHub as well.