Clinical decision support based on Bevacizumab cancer trials and pushing the limitations of advanced LLMs

An exhaustive study was needed to test the limits of leading Large Language Models (LLMs) using numerous real-world clinical trial outcomes. It was also necessary to provide extensive hallucination studies based on both extracting main points and providing novel AI clinical decision support. Here, 100 Bevacizumab cancer therapy articles representing over 900K words were summarized by the 3.5 Sonnet model into 49K words, which completed a detailed and complex problem of several cancer and study types to press the capabilities of the ChatGPT o1 reasoning model. Report summaries in general followed an effective format, with guardrails to de-identify patient information and numerical data sources attributions to ground the output to the input. The main takeaway was that both LLMs typically remained in-context with the input data when more structured prompts were used, while precise quotations and author name citations were less prominent. These errors were likely due to LLM pressure towards achieving coherence vs. exact recall based on manufacturer inference-time compute settings. Overall, ChatGPT o1 provided state-of-the-art evidence-based Bevacizumab insights regarding clinical efficacy across indications, dosing recommendations, combination therapies, and biomarker-driven selections.