200 Annotated Developer Human Errors from GitHub

Software Engineers' Human Errors

This dataset contains 200 GitHub comments with manual human error annotations, released as part of the following publication:

• Benjamin S. Meyers. Human Error Assessment in Software Engineering. Rochester Institute of Technology. 2023.

Included Files

The "developer_human_errors.csv" file contains the full dataset of 200 software defect descriptions annotated with human error types (slips, lapses, mistakes) and T.H.E.S.E. categories.

CSV Fields

• ID: Unique identifier for the comment.
• SOURCE: Whether this comment originates from a commit, issue, or pull request.
• COMMENT_URL: The URL linking to the comment.
• COMMENT_TEXT: The raw comment text.
• HUMAN_ERROR_TYPE: Whether the software defect described is a slip, lapse, or mistake.
• THESE_V4_ID: Manually assigned T.H.E.S.E. category with labels corresponding to Version 4 of T.H.E.S.E.
• THESE_NAME: Name corresponding to manually assigned T.H.E.S.E. category.

Annotation Details

Human error types span slips, lapses, and mistakes from James Reason's Generic Error Modelling System (GEMS):

• Slips: Failures of attention.
• Lapses: Failures of memory.
• Mistakes: Failures of planning.

T.H.E.S.E. categories are summarized below:

• S01: Typos & Misspellings
• S02: Syntax Errors
• S03: Overlooking documented Information
• S04: Multitasking Errors
• S05: Hardware Interaction Errors
• S06: Overlooking Proposed Code Changes
• S07: Overlooking Existing Functionality
• S08: General Attentional Failure
• L01: Forgetting to Finish a Development Task
• L02: Forgetting to Fix a Defect
• L03: Forgetting to Remove Development Artifacts
• L04: Working with Outdated Source Code
• L05: Forgetting an Import Statement
• L06: Forgetting to Save Work
• L07: Forgetting Previous Development Discussion
• L08: General Memory Failure
• M01: Code Logic Errors
• M02: Incomplete Domain Knowledge
• M03: Wrong Assumption Errors
• M04: Internal Communication Errors
• M05: External Communication Errors
• M06: Solution Choice Errors
• M07: Time Management Errors
• M08: Inadequate Testing
• M09: Incorrect/Insufficient Configuration
• M10: Code Complexity Errors
• M11: Internationalization/String Encoding Errors
• M12: Inadequate Experience Errors
• M13: Insufficient Tooling Access Errors
• M14: Workflow Order Errors
• M15: General Planning Failure

Contact

Please contact Benjamin S. Meyers (email) with questions about this data and its collection.

Acknowledgments

Collection of this data has been sponsored in part by the National Science Foundation (grant 1922169), by the NSA Science of Security Lablet program (grant H98230-17-D-0080/2018-0438-02), and by a Department of Defense DARPA SBIR program (grant 140D63-19-C-0018).