JML-BugDB Dataset & JavaMLBugDetective Replication Package

Version 3. Supersedes version 2 (distributed as package v1.1) and version 1 (package v1.0). Prepared for the revised submission to Science of Computer Programming. Figures from earlier versions should not be cited.

Manual validation re-run with a two-rater protocol. Estimated SZZ label precision is now 49.5% (197/398) with inter-rater agreement Cohen's κ = 0.799, replacing the 29.2% reported in versions 1 and 2, which came from a single-rater audit with no documented agreement statistic. Per-project precision: Kafka 57.1%, Gson 48.5%, Commons-IO 42.9%. Both raters' blind passes, the disagreement-resolution record for the 10 conflicting cases, and a per-instance false-positive taxonomy of all 201 false positives are included.

The labelling question changed, so labels are not directly comparable across versions. The is_valid_fix column has been renamed is_true_bic. The earlier audit recorded whether a candidate was itself a valid fix; the current audit records whether the candidate introduced the defect that its traced fix later repaired. These are different questions, and the resulting labels cannot be compared row by row with earlier versions.

A central claim has been withdrawn. Earlier versions of the paper argued that feature-engineering quality has a stronger influence on predictive performance than label purity. A label-noise sensitivity experiment, included here, does not support that claim: adding 20 percentage points of label noise erases more MCC than the entire static-to-hybrid feature-engineering gain, in every project. The claim has been removed from the manuscript and replaced with the measured finding that process features hold a consistent advantage over static features at the observed noise level, while performance remains sensitive to further label degradation.

Ablation and significance results re-derived from logged runs. All values now come from 270 independently seeded 10-fold cross-validation runs (30 seeds × 3 projects × 3 feature sets), with per-seed raw results, runner scripts and exact seeds included. The reported Hybrid-over-Static improvement on Commons-IO is 55%, not the 128% stated previously; Static-only MCC on Commons-IO is +0.166, not negative. Statistical testing is a Wilcoxon signed-rank test paired by seed with Cliff's delta and Holm-Bonferroni correction over nine comparisons per metric.

MCC replaces F1 as the primary metric, and class prevalence, trivial baselines and confusion matrices are now reported. Prevalence differs by a factor of ~2.7 across the three projects (55.4% / 44.9% / 20.9%), so F1 is not comparable across them; all trivial baselines have MCC = 0.

The reported pipeline does not use SMOTE. A direct comparison, included here, shows no benefit in any project and a measurable cost on Commons-IO. SMOTE remains available as a configuration option in the framework.

Runtime and energy figures are measured, not extrapolated: 30.6 minutes wall-clock across all three projects for the reported configuration, with per-phase timings and logs.

New in this version: the label-noise sensitivity sweep; an AGE-feature ablation; confusion matrices and specificity per project and feature set; a Process-versus-Static significance test; a read-only exporter for the SZZ fix-to-BIC mapping, which the original pipeline did not persist; and a replication configuration annotated with the exact settings used for the reported results, distinguishing them from the framework's defaults.

Not carried forward: single-run predecessors of the 30-seed result files (ablation_study_results.csv, statistical_significance.csv), a Friedman test and bootstrap confidence intervals superseded by the Wilcoxon/Cliff's-delta analysis, a single-run cross-project results file whose AUC-ROC columns the current manuscript does not use, and an aggregate feature-importance file replaced by a per-project one. All remain available, unchanged, in the earlier Zenodo versions.

Dataset unchanged: JML-BugDB v1.0, 91,633 instances.

Replication package for the paper "An Initial Empirical Evaluation of Evolutionary Context Modeling for Defect Prediction under Quantified Label Noise in Enterprise Java Systems" (submitted to Science of Computer Programming; previously titled "Beyond the Gold Standard: Validating Evolutionary Context Modeling...").

Contains:

• JML-BugDB v1.0 dataset — 91,633 file revisions from Apache Kafka, Google Gson and Apache Commons-IO, covering 25,480 commits and 17 years of development history, in CSV and ARFF form.
• Manual validation data — a 398-instance two-rater audit of the SZZ labels (estimated precision 49.5%, Cohen's κ = 0.799), including both raters' blind passes, the disagreement-resolution record, and a per-instance false-positive taxonomy.
• Experimental results — 270 logged, independently seeded ablation and significance runs; a label-noise sensitivity sweep; an AGE-feature ablation; a cost-ratio sensitivity sweep; a SMOTE comparison; permutation feature importance; measured runtime and energy figures with per-phase logs.
• JavaMLBugDetective framework source code and the experiment runners needed to reproduce every reported result.
• Replication guide with step-by-step instructions.