Correlation Between Visual Noise Robustness and Performance Degradation in Multimodal LLMs Across Medical and General Tasks

Recently, large language models (LLMs) have taken the spotlight in natural language processing. Further, integrating LLMs with vision enables the users to explore emergent abilities with multimodal data. Visual language models (VLMs), such as LLaVA, Flamingo, or CLIP, have demonstrated impressive performance on various visio-linguistic tasks. Consequently, there are enormous applications of large models that could be potentially used in the biomedical imaging field. Along that direction, there is a lack of related work to show the ability of large models to diagnose the diseases. In this work,

Research goal: How does the robustness of state-of-the-art multimodal LLMs to visual noise correlate with their performance degradation on medical decision-making benchmarks versus general vision-language tasks?

Autonomous synthesis report generated by SOVEREIGN Research Kernel. Tribunal consensus score: 8.4/10.