Conference paper Open Access

Alleviating Digitization Errors in Named Entity Recognition for Historical Documents

Boros, Emanuela; Hamdi, Ahmed; Pontes, Elvys Linhares; Adrian Cabrera-Diego, Luis; Moreno, Jose G.; Sidere, Nicolas; Doucet, Antoine

This paper tackles the task of named entity recognition (NER) applied to digitized historical texts obtained from processing digital images of newspapers using optical character recognition (OCR) techniques. We argue that the main challenge for this task is that the OCR process leads to misspellings and linguistic errors in the output text. Moreover, historical variations can be present in aged documents, which can impact the performance of the NER process. We conduct a comparative evaluation on two historical datasets in German and French against previous state-of-the-art models, and we propose a model based on a hierarchical stack of Transformers to approach the NER task for historical data. Our findings show that the proposed model clearly improves the results on both historical datasets, and does not degrade the results for modern datasets.

Files (409.2 kB)
Name Size
alleviating_digitization_errors_in_named_entity_recognition_for_historical_documents.pdf
md5:d78c27b3f211c879de332ef4dac6b28e
409.2 kB Download
61
66
views
downloads
All versions This version
Views 6161
Downloads 6666
Data volume 27.0 MB27.0 MB
Unique views 5252
Unique downloads 6565

Share

Cite as