Morphology data for 22 languages

Most people will want to download only the train_data.zip file. This contains folders named after language ISO codes. Under each language, you will find separate folders for the lemmatization, analysis and generation tasks. Each one of these folders has source and target files for train, val and test. There is also a pred.txt file that has the predictions of the baseline system. If you need more control over how these datasets are created, keep on reading.

To create the dataset from scratch, run the data_formatter.py. The morphological data is stored in the fst.zip, if you want to download that and skip some parts of making the data from scratch, extract it to the same folder as the data_formatter.py.

Cite:

Hämäläinen, M., Partanen, N., Rueter, J., & Alnajjar, K. (2021). Neural Morphology Dataset and Models for Multiple Languages, from the Large to the Endangered. In Proceedings of the 23rd Nordic Conference on Computational Linguistics (NoDaLiDa 2021)