The unsupervised deployment gap in generative AI for mental health

Evidence about artificial intelligence in mental health and the deployment of artificial intelligence in mental health concern two different objects, and the first is routinely cited to justify the second. The evidence base is forty randomized trials pooling to g = 0.31 for depressive symptoms and g = 0.28 for anxiety, with 35 of 39 trials at high risk of bias, significant publication bias, and only 8 trials testing a generative system. The one randomized trial of a fully generative therapy chatbot enrolled 210 adults for four weeks, ran on Falcon-7B and LLaMA-2-70B, had clinicians review every response, and excluded active suicidality, mania and psychosis, turning away 215 people for suicide risk while enrolling 210 in total. Recomputing that trial's effect sizes from its own published means and standard deviations gives standardized mean differences of 0.45 to 0.62 against reported values of 0.63 to 0.90.

 

Deployment is three orders of magnitude larger. The operator of the most widely used general assistant estimates that in a given week 0.15% of active users have conversations containing explicit indicators of suicidal planning or intent and 0.07% show possible signs of psychosis or mania. Against 800 million weekly active users that describes roughly a million people every week.

 

This paper proposes a taxonomy of four deployment classes defined by design intent, regulatory claim, population gate, supervision and exposure, and shows that evidentiary strength and population exposure run in opposite directions across them. It argues that sycophancy in this setting is a problem of reward specification rather than a defect of alignment: therapeutic benefit cannot be observed inside a conversation, user approval can, and expert clinicians agree on whether a response is desirable only 71% to 77% of the time. It identifies two regulatory inversions in the four United States statutes passed since March 2025, and proposes psychovigilance: version identity, reporting attached to function rather than claim, independent access to logs, and outcome measurement at the level of whole conversations.