General segmentation model for print and handwriting
Description
BLLA base model
This is the default segmentation model shipped with the kraken ATR engine. It replaces the previous base model trained on cBAD 2019 alone and has been retrained from scratch on a considerably larger and more diverse corpus of ~11200 pages aggregated from six datasets.
Uses
The model performs reasonably well on most non-fragmentary handwritten and machine-printed document pages of moderate complexity. If out of the box performance is unsatisfactory or more fine-grained segmentation taxonomies are desired it can also serve as a good foundation model for fine-tuning.
Training Data
The training corpus aggregates six publicly available datasets. Pages were split 11,271 / 1,000 / 1,000 into training, validation, and test sets at random.
| Dataset | Source | Train | Val | Test | Total | | --- | --- | ---: | ---: | ---: | ---: | | OCR-D-LAYOUT-GT-DTA | 10.5281/zenodo.19348674 | 3,741 | 346 | 326 | 4,413 | | Burchards Dekret Digital | 10.5281/zenodo.10882817 | 3,035 | 246 | 257 | 3,538 | | cBAD 2019 | 10.5281/zenodo.3568023 | 2,582 | 209 | 230 | 3,021 | | Muharaf | 10.5281/zenodo.11492215 | 1,011 | 101 | 104 | 1,216 | | CLLG | gitlab.inria.fr/almanach/cllg | 460 | 47 | 40 | 547 | | Baybars | gitlab.huma-num.fr/lipa/baybars | 442 | 51 | 43 | 536 | | Total | | 11,271 | 1,000 | 1,000 | 13,271 |
The exact splits are deposited alongside the model as train.lst, val.lst,
and test.lst. Each line is the path of a PAGE or ALTO file relative to the
root of the combined corpus. The datasets are freely licensed and can be
obtained from the sources above.
Region and line types in the training dataset have been merged into single classes.
Biases and Limitations
The goal of this training run was to take advantage of the superior accuracy of models trained with the recently implemented hybrid CE+Dice loss but also to improve accuracy on machine-printed documents with close writing, images taken with handheld devices, and non-Latin material. Our tests indicate that this was generally successful, nevertheless the model still exhibits some of the known weaknesses.
The training corpus consists exclusively of Latin, Arabic, and Greek script documents with relatively simple layouts. Accuracy on scripts and layouts dissimilar to those three is therefore expected to be uneven. Some features of pages where segmentation accuracy is frequently unsatisfactory:
- Tilted and vertical writing
- Writing embedded in decoration
An idiosyncracy of the model is that no matter the position of the calligraphic/typographic baseline (top-, center-, or baseline) a baseline will be detected. Hebrew script lines will therefore be detected with base- not toplines. This is generally not an issue for recognition models that have been trained with the same line position.
How to Get Started with the Model
Install the kraken ATR package following the instruction
here. A quick start explaining the use of the segmenter
can also be found there.
Training
The model was trained from scratch with kraken's default BLLA architecture for 20 epochs on the 11,271 page training split, with data augmentation enabled. The optimizer was AdamW at a learning rate of 2e-4 with a weight decay of 1e-5, gradient clipping at 1.0, and a cosine schedule with 2,000 warmup steps decaying to a minimum learning rate of 2e-5.
Evaluation
Evaluated with ketos segtest on the held-out 1,000 page test split. The line
evaluation uses kraken's adaptation of the cBAD baseline evaluation metric:
| Precision | Recall | F1 | | ---: | ---: | ---: | | 0.964 | 0.947 | 0.950 |
Pixel accuracy and IOU for the text region:
| Pixel accuracy | IOU | | ---: | ---: | | 0.965 | 0.929 |
Acknowledgements
Training of this model was funded by the European Union under Grant Agreement No.~101132163 (ATRIUM). Views and opinions expressed are those of the authors only and do not necessarily reflect those of the European Union.
Citation
If you use this model, please cite kraken and if possible credit the dataset providers linked in the front matter and the table above.
BibTeX:
@inproceedings{10.1007/978-3-032-04624-6_26, author = {Kiessling, Benjamin}, title = {Version 5 of the Kraken ATR Engine for the Humanities}, year = {2025}, isbn = {978-3-032-04623-9}, publisher = {Springer-Verlag}, address = {Berlin, Heidelberg}, doi = {10.1007/978-3-032-04624-6_26}, booktitle = {Document Analysis and Recognition – ICDAR 2025: 19th International Conference, Wuhan, China, September 16–21, 2025, Proceedings, Part III}, pages = {443–458}, numpages = {16}, location = {Wuhan, China} }