Published April 5, 2025 | Version v1

A Pilot Study on Fine-tuning Named Entity Recognition for Clinical Tag Extraction Using Pretrained Language Models: The Tut-All Experience

  • 1. Centre of Excellence for Data Science, Artificial Intelligence and Modelling
  • 2. ROR icon University of Hull
  • 3. Tut-All Software GmbH
  • 4. Centre for Biomedicine, Hull York Medical School

Description

The standard procedure to extract data from medical records is still a laborious process where research analysts examine a wide range of information sources. Hence, there is a strong need to improve automatic data extraction techniques. This study performed fine-tuning of the pretrained language models (PLMs)—BioElectra, PubMedBert, BioLinkBert, and Clinical Longformer—for named entity recognition (NER) in clinical texts. The clinical texts were extracted from records using the Tut-All software and were comprised of abstracts and methods sections. For performance analysis the mean of the F1-score for each named entity tag was used across different hyperparameter optimizations. PubMedBert achieved the best performance results with a mean F1-score of 0.638 and an output range of 0.3. This performance was achieved after training with a learning rate and a weight decay of 5×10-5 and 3×10-3 respectively, employing default HuggingFace hyperparameters. Notably, PubMedBert exhibited superior performance on minority entity types, addressing a key challenge in real-world data applications. The optimized model not only expedites information retrieval, but also enhances the overall quality of extracted data, marking a valuable contribution to the data extraction data from medical records.

Files

Final_AdeyemiSYMCOMP2025.pdf

Files (472.3 kB)

Name Size
md5:45aaad59fa9a886906236f62a0a47513
472.3 kB Preview Download

Additional details

Identifiers

ISBN
978-989-99410-8-3