Dataset Open Access

WiLI-2018 - Wikipedia Language Identification database

Thoma, Martin

WiLI-2018, the Wikipedia language identification benchmark dataset, contains 235000 paragraphs of 235 languages. The dataset is balanced and a train-test split is provided.

See "The WiLI benchmark dataset for written language identification" paper (soon on arXiv) for more information.

Files (62.4 MB)
Name Size
wili-2018.zip
md5:3dc5bd41587811ad6b0d04ae2f235f84 		62.4 MB Download
1,185
468
views
downloads
All versions This version
Views 1,1851,185
Downloads 468468
Data volume 29.2 GB29.2 GB
Unique views 1,0631,063
Unique downloads 399399
More info on how stats are collected.
Indexed in
Publication date:
January 7, 2018
DOI:
10.5281/zenodo.841984

Zenodo DOI Badge

DOI 

10.5281/zenodo.841984

Markdown 

[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.841984.svg)](https://doi.org/10.5281/zenodo.841984)

reStructedText 

.. image:: https://zenodo.org/badge/DOI/10.5281/zenodo.841984.svg
   :target: https://doi.org/10.5281/zenodo.841984

HTML 

<a href="https://doi.org/10.5281/zenodo.841984"><img src="https://zenodo.org/badge/DOI/10.5281/zenodo.841984.svg" alt="DOI"></a>

Image URL 

https://zenodo.org/badge/DOI/10.5281/zenodo.841984.svg

Target URL 

https://doi.org/10.5281/zenodo.841984

Communities:
License (for files):
ODC Open Database License v1.0

Versions

Version 1.0.0 10.5281/zenodo.841984 Jan 7, 2018
Cite all versions? You can cite all versions by using the DOI 10.5281/zenodo.841983. This DOI represents all versions, and will always resolve to the latest one. Read more.

Share

Cite as

Export