Outlier Failure Analysis: A Data-Centric Approach to Hard-Negative Mining in YOLO

Object detection models still fail on unrepresented backgrounds after fine-tuning, and finding the samples that cause those failures by hand does not scale. We describe the OutlierFailureAnalyzer, a post-training module at step 13 of the train_service2 worker pipeline that turns validation inferences into training corrections. The module mines high-confidence false positives, false negatives, and localization errors, stores them as structured hard negatives, and feeds them back into the next training iteration. We evaluate it on the public COCO128 subset under a leakage-free protocol: a held-out split that never enters fine-tuning, four A/B arms, and two ablations. Fine-tuning a frozen-backbone YOLOv8n on the 94-image working set retains held-out mAP50 but raises false positives. Reintegrating the mined hard-negative crops pushes the high-confidence false-positive rate from 8.33% to 2.00% in this single-seed run---below the pretrained baseline---at a mAP50 cost under 0.02. The gain is directional, not established: on 32 held-out images the Wilson 95% intervals ([3.6%, 18.1%] and [0.4%, 10.5%]) overlap, and neither Fisher's exact test nor a seeded permutation test reaches significance. Naive false-negative rebalancing points the other way, lifting the same rate to 12.20% while recovering no recall. We analyze those statistical limits and the engineering constraints a production feedback loop must respect.