Detecting Outdated Documentation in Software Repositories

Background: Outdated documentation is a pervasive problem in software development, preventing effective use of software and misleading users and developers alike. We posit that one reason why documentation becomes out of sync so easily is that developers are not aware of when their source code modifications render the documentation obsolete.

Aims: Ensuring that the documentation is in sync with the source code at all times takes considerable effort, especially for large projects. To address this situation, we propose an approach that can automatically detect outdated documentation in software repositories to help developers keep their documentation up to date.

Method: Our approach detects outdated documentation by looking for remaining code element references still referred to from the repository's documentation after all code element instances have been deleted in the source code.

Results: We analysed over 2,000 GitHub projects and found at least one outdated reference in most of the projects. We submitted GitHub issues to 15 real-world projects containing outdated references detected by our approach, some of which have already led to documentation fixes.

Conclusions: Our results show that outdated documentation affects the majority of the projects. Using our publicly available implementation, developers can scan their GitHub projects for outdated documentation. Our work is a first step toward keeping documentation in software repositories up to date.