Info: Zenodo’s user support line is staffed on regular business days between Dec 23 and Jan 5. Response times may be slightly longer than normal.

Published August 25, 2023 | Version 2.0.0
Software Open

ESEC/FSE'23 Artifact for "Copiloting the Copilots: Fusing Large Language Models with Completion Engines for Automated Program Repair"

  • 1. University of Illinois at Urbana-Champaign

Description

Artifact (Docker Container) for our ESEC/FSE'23 Paper: "Copiloting the Copilots: Fusing Large Language Models with Completion Engines for Automated Program Repair"

NOTE: Some links in "README-Artifact.md" from this upload may not work. Please refer to our GitHub repository or "/root/Repilot/README-Artifact.md" within the Docker image.

During Automated Program Repair (APR), it can be challenging to synthesize correct patches for real-world systems in general-purpose programming languages. Recent Large Language Models (LLMs) have been shown to be helpful “copilots” in assisting developers with various coding tasks, and have also been directly applied for patch synthesis. However, most LLMs treat programs as sequences of tokens, meaning that they are ignorant of the underlying semantics constraints of the target programming language. This results in plenty of statically invalid generated patches, impeding the practicality of the technique. Therefore, we propose Repilot, a framework to further copilot the AI “copilots” (i.e., LLMs) by synthesizing more valid patches during the repair process. Our key insight is that many LLMs produce outputs autoregressively (i.e., token by token), resembling human writing programs, which can be significantly boosted and guided through a Completion Engine. Repilot synergistically synthesizes a candidate patch through the interaction between an LLM and a Completion Engine, which 1) prunes away infeasible tokens suggested by the LLM and 2) proactively completes the token based on the suggestions provided by the Completion Engine. Our evaluation on a subset of the widely-used Defects4j 1.2 and 2.0 datasets shows that Repilot fixes 66 and 50 bugs, respectively, surpassing the best-performing baseline by 14 and 16 bugs fixed. More importantly, Repilot is capable of producing more valid and correct patches than the base LLM when given the same generation budget

For more information, please look into our GitHub repository: https://github.com/ise-uiuc/Repilot.

Files

README-Artifact.md

Files (11.7 GB)

Name Size Download all
md5:44500bd10dddb06e2929252c853eb01c
10.9 kB Preview Download
md5:ea5271cfbce6f3dd839354413726d1e0
11.7 GB Download

Additional details

Related works

Is cited by
Conference paper: 10.1145/3611643.3616271 (DOI)