There is a newer version of the record available.

Published October 17, 2026 | Version v5.0

cpath-triage: diagnostic analysis of verbalized confidence in a medical vision-language model

  • 1. Department of Computer and Information Science and Engineering, University of Florida, Gainesville, FL, U.S.A.
  • 2. Department of Computer Science and Engineering, JK Lakshmipat University, Jaipur, India
  • 3. Department of Data Science and Engineering, Manipal University Jaipur, Jaipur, India
  • 4. Department of Computer Applications, Manipal University Jaipur, Jaipur, India

Description

Evaluation and analysis code, plus the prediction artifacts behind every table and figure, for a diagnostic study of verbalized confidence in MedGemma-27b-it on colorectal H&E histopathology patches. Covers zero-shot classification, temperature scaling and calibration metrics, selective-prediction routing with explicit tie handling, a design-effect sensitivity analysis in place of clustered intervals, and K=5 prompt-variant consistency routing. Version 4.0 added a cohort-weighted re-analysis of the K=5 subsets, a wording-only prompt family, image-removal, image-swap and pixel-shuffle controls, and a PLIP image-text baseline. Version 5.0 adds the wording-only family on the external cohort, each prompt variant's own stated confidence, Platt and isotonic calibration maps with a Brier decomposition, agreement against the number of prompts, operating points with measured query cost, per-class results under each pilot prompt, a QuiltNet image-text baseline, and the per-call logs of the runs that kept them.

Notes

If you use this software or the prediction artifacts it contains, please cite both the software and the paper below.

Files

cpath-triage-v5.0-code-and-artifacts.zip

Files (23.6 MB)

Name Size
md5:abe5e38f25f537e2815c4f7903ceeb07
23.6 MB Preview Download

Additional details

Related works