Automation Bias in Clinical AI: Why Explainability Is a Safety Requirement, Not a Feature

Artificial intelligence tools deployed in clinical settings frequently present risk scores or predictions without explanatory context. This paper argues that unexplained predictions do not merely fail to help physicians reason — they actively distort reasoning by amplifying automation bias: the well-documented tendency to accept automated recommendations without critical appraisal. Drawing on cognitive psychology, signal detection theory, and dual-process models of clinical decision-making, we identify two mechanisms by which opaque predictions amplify automation bias — System 1 anchoring and trust calibration failure — and describe a third, structural mechanism: asymmetric feedback loops that systematically cause physicians to overestimate AI reliability over time. We derive three design principles for clinical AI that mitigate these mechanisms without reducing predictive utility: mandatory prediction explanation, an explicit false-positive pathway, and bounded intervention scope. These principles are illustrated through AdherenceAI, a clinical decision support tool for non-adherence risk in type 2 diabetes, which operationalises each principle in its architecture. The central argument is that in time-pressured clinical environments, explainability is not a regulatory formality or a usability enhancement. It is infrastructure for cognitively safe AI deployment.