Replication Kit - What the Fix? A Study of ASATs Rule Documentation

Contains all artefacts for our publication: What the Fix? A Study of ASATs Rule Documentation @ ICPC 2024

Content

• survey: Archive from LimeSurvey, rules annotated used and raw results
• taxonomy_validation: Rules annotated by the authors to validate the taxonomy and notebook computing kappa values
• website: Build of the companion webpage (currently hosted at https://icpc2024-asats.github.io)

Paper abstract
Automatic Static Analysis Tools (ASATs) are widely used by software developers to diffuse and enforce coding practices. Yet, we know little about the documentation of ASATs, despite it being critical to learn about the coding practices in the first place. We shed light on this through a two-phase study. First, we analyze the documentation of more than 100 rules of 16 ASATs for multiple programming languages, and distill a taxonomy of the goals of the documentation and its types of contents. Then, we conduct a survey to assess the effectiveness of the documentation in terms of its goals and types of contents. We highlight opportunities for improvement in ASAT documentation.