Conference paper Open Access

Alleviating Digitization Errors in Named Entity Recognition for Historical Documents

Boros, Emanuela; Hamdi, Ahmed; Pontes, Elvys Linhares; Adrian Cabrera-Diego, Luis; Moreno, Jose G.; Sidere, Nicolas; Doucet, Antoine

This paper tackles the task of named entity recognition (NER) applied to digitized historical texts obtained from processing digital images of newspapers using optical character recognition (OCR) techniques. We argue that the main challenge for this task is that the OCR process leads to misspellings and linguistic errors in the output text. Moreover, historical variations can be present in aged documents, which can impact the performance of the NER process. We conduct a comparative evaluation on two historical datasets in German and French against previous state-of-the-art models, and we propose a model based on a hierarchical stack of Transformers to approach the NER task for historical data. Our findings show that the proposed model clearly improves the results on both historical datasets, and does not degrade the results for modern datasets.

Files (409.2 kB)
Name Size
409.2 kB Download
All versions This version
Views 6161
Downloads 6666
Data volume 27.0 MB27.0 MB
Unique views 5252
Unique downloads 6565


Cite as