Bridging the Medical Knowledge Gap: Investigating the Efficacy of 4-bit Instruction-Tuning on Llama-3-8B for Clinical Reasoning

In this publication, we introduce a large language model (LLM) called Bio-Llama-3-Expert that has been designed for difficult clinical biomedical reasoning tasks. In order to do so, we performed parameter-efficient training methods such as quantized low-rank adaptation to fine-tune the Meta Llama-3-8B LLM - which runs on a single NVIDIA T4 graphics processing unit (GPU) with 16GB of video RAM (VRAM). The performance improvements experienced through our method of fine-tuning were substantial and the final version of our model proved to be highly effective in completing medical reasoning tasks, while being able to operate on a typical consumer computer. Additionally, our best result demonstrates that Bio-Llama-3-Expert achieved a score of 80% accuracy in answering clinical biomedical questions through expert various few-shot prompting methods, resulting in a 28% increase over the original baseline result for clinical biomedical reasoning tasks.

This is a preprint made publicly available via Zenodo.