Reliability Failure Modes of VLM-Based Open-Vocabulary Object Detection: A Narrative Review of Region-Text Alignment and Prompt Sensitivity

Open-vocabulary object detection (OVD) extends object detection beyond fixed category labels by allowing models to detect objects specified through class names, phrases, or natural-language descriptions. Vision-language models (VLMs) have enabled this transition by aligning visual and textual representations and supporting text-conditioned detection at inference time. However, strong benchmark performance does not guarantee reliable behavior. VLM-based open-vocabulary detectors remain vulnerable to failures that arise when image-level language supervision is transferred to region-level localization and when small changes in prompt wording alter detector output. This narrative review synthesizes representative work on CLIP, RegionCLIP, Detic, OWL-ViT, GLIP, Grounding DINO, OWLv2, YOLO-World, region-alignment methods, and recent prompt-sensitivity studies. Complementing broad task- and method-centered OVD surveys, the review uses reliability failure mechanisms as the organizing lens and jointly examines region-text misalignment and prompt-conditioned instability. The main failure patterns include incorrect region-text association, missed detections, false positives, weak localization, synonym instability, and inconsistent responses to semantically related prompts. Conventional metrics such as AP and mAP remain necessary but insufficient because they do not directly measure prompt robustness, alignment stability, or unsupported-detection risk. To operationalize reliability analysis, the review synthesizes complementary evaluation of selection consistency, box consistency, confidence stability, prompt-performance dispersion, generic-to-specific behavior, and absent-object false positives. Future progress should emphasize region-level alignment, transparent prompt reporting, uncertainty estimation, and qualitative failure analysis so that OVD systems become not only capable but also stable, interpretable, and practically reliable.