Wikidata Lemmatization Dataset

The Wikidata Lemmatization Dataset was collected using the following SPARQL query: https://w.wiki/9TwH

Languages included in the dataset:

• Akkadian : AKK (Q35518)
• Arabic : AR (Q13955)
• Czech : CS (Q9056)
• German : DE (Q188)
• English : EN (Q1860)
• French : FR (Q150)
• Hebrew : HE (Q9288)
• Hittite : HIT (Q35668)
• Italian : IT (Q652)
• Russian : RU (Q7737)
• Sumerian : SUX (Q36790)
• Turkish : TR (Q256)

The choice of languages to include have to do with a collection of primary and secondary source documents which we have digitized (OCR) and are using as references for the FactGrid Cuneiform project. The resulting lexemes for each language are shared in CSV with the file names references each language, their Wikidata Q-ids, the number of lexemes at that date, and the date of access (MM_YYYY).

The format of each CSV includes the following fields:

1. lexeme : the Wikidata lexeme id (L-id)
2. lexemeLabel : the label assigned to the lexeme in Wikidata
3. lexical_category : the Wikidata Q-item for the part of speech
4. lexical_categoryLabel : the label assigned to the lexical category (e.g. noun, verb, adjective, etc.)

This dataset will be updated periodically using standard version control.