Published January 1, 2026 | Version v1

Explainable Vision- Language AI for Semantic Understanding of Medical Images and Diagnostic Text

Description

Combining vision and language clinical information is important for intelligent diagnosis, but most medical vision-language models are black-boxed in the process of reasoning. This paper proposes an explainable vision-language framework for semantic analysis of medical images along with their diagnostic texts. The framework consists of a multi-scale vision encoder, a clinical text encoder, and a cross-modal alignment layer with an explanation generation pipeline. The model provides diagnostic predictions together with spatial evidence from the image and token-level evidence from text, allowing users to analyze the reasoning process behind every prediction. We experiment with two medical image datasets with their diagnostic texts and compare the framework with other unimodal and multimodal baselines in terms of diagnostic ability and explanation faithfulness. Our experiments show that adding explanation supervision does not hurt the diagnostic ability while greatly enhancing the clinical plausibility of explanations.

Files

IJSET_V14_issue5_214.pdf

Files (469.8 kB)

Name Size
md5:5d405b556b1fcf6f9726e3e0663fc019
469.8 kB Preview Download

Additional details