Automated Surgical Report Generation Using In-context Learning with Scene Labels from Surgical Videos

We propose a method for generating surgical reports from surgical video scene labels and demonstrate the effectiveness of In-context Learning (ICL) in this process. Writing surgical reports is a significant burden for surgeons. Utilizing the open-source language model Llama 3 (8b), we generate surgical reports from scene labels of surgical videos through few-shot learning, comparing the performance of 1-shot, 2-shot, and 3-shot scenarios. Gynecologists wrote reference surgical reports for ten videos, and the generated reports were evaluated based on the number of errors compared to these references. The results indicate that increasing the number of shots reduces errors in the generated reports, confirming the effectiveness of ICL in surgical report generation. This approach has the potential to alleviate the documentation workload for surgeons, improving efficiency and accuracy in medical reporting.