Published August 25, 2025
| Version v1.0.1
Software
Open
Replication Package: A Comprehensive Analysis of ChatGPT's Influence on Pull Request Outcomes
Authors/Creators
Description
The rapid adoption of large language models (LLMs) like
ChatGPT has introduced new dynamics in software development, particularly within pull request workflows. While prior research has examined the quality of AI-generated code, little is known about how developers actually use these suggestions in real-world collaboration. We analyze 338 pull requests from 255 GitHub repositories containing self-admitted ChatGPT usage, including 645 AI-generated snippets and 3,486 developer-authored patches. We introduce PatchTrack, a tool that classifies whether ChatGPT patches were applied, not applied, or not suggested, enabling fine-grained analysis of AI-assisted decisions. Full adoption of ChatGPT code is rare: the median integration rate was 25%. A qualitative analysis of 89 pull requests with integrated patches revealed recurring patterns of structural integration, selective extraction, and iterative refinement, showing that developers typically treat ChatGPT’s output as a starting point rather than a final implementation.Files
Files
(41.4 MB)
| Name | Size | Download all |
|---|---|---|
|
md5:efd2598a694784f096b591a1d0073846
|
41.4 MB | Download |
Additional details
Software
- Repository URL
- https://github.com/unlv-evol/PatchTrack
- Programming language
- Python , Jupyter Notebook
- Development Status
- Active