Consumer AI as Health Plan Decision Support: A benchmark of LLM plan recommendations against a claims-based pricing engine

Consumers increasingly ask AI chatbots to recommend health insurance plans, yet little is known about whether they estimate costs accurately or select the right plan. We benchmark the July 2026 consumer AI lineup from OpenAI and Anthropic on a realistic plan-selection task: five ACA-compliant plans, twelve claims-anchored member profiles, four progressively richer levels of consumer-provided health information, and a claims-based pricing engine as the reference standard. Across 1,680 AI responses, we evaluate cost estimation, plan recommendation accuracy, optimization objective, and response consistency. Current frontier models recommend a near-optimal plan in roughly half of the responses, up from one-third for a previous-generation model. Most error is attributable to the difficulty of mapping consumer-described care to the full-service mix and market-level prices, rather than to plan-selection logic. Providing more detailed consumer information yields diminishing returns, whereas supplying prices raises correct recommendations to 85–95%. These findings suggest that the primary constraint on AI-assisted plan selection is shifting from model reasoning to the quality and availability of insurance data.