Dataset Open Access

WiLI-2018 - Wikipedia Language Identification database

Thoma, Martin

WiLI-2018, the Wikipedia language identification benchmark dataset, contains 235000 paragraphs of 235 languages. The dataset is balanced and a train-test split is provided.

See "The WiLI benchmark dataset for written language identification" paper (soon on arXiv) for more information.

Files (62.4 MB)
Name Size
wili-2018.zip
md5:3dc5bd41587811ad6b0d04ae2f235f84 		62.4 MB Download
5,186
3,384
views
downloads
All versions This version
Views 5,1865,184
Downloads 3,3843,384
Data volume 211.2 GB211.2 GB
Unique views 4,5524,551
Unique downloads 2,2002,200
More info on how stats are collected.
Indexed in
Publication date:
January 7, 2018
DOI:
10.5281/zenodo.841984

Zenodo DOI Badge

DOI 

10.5281/zenodo.841984

Markdown 

[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.841984.svg)](https://doi.org/10.5281/zenodo.841984)

reStructedText 

.. image:: https://zenodo.org/badge/DOI/10.5281/zenodo.841984.svg
   :target: https://doi.org/10.5281/zenodo.841984

HTML 

<a href="https://doi.org/10.5281/zenodo.841984"><img src="https://zenodo.org/badge/DOI/10.5281/zenodo.841984.svg" alt="DOI"></a>

Image URL 

https://zenodo.org/badge/DOI/10.5281/zenodo.841984.svg

Target URL 

https://doi.org/10.5281/zenodo.841984

Communities:
License (for files):
Open Data Commons Open Database License v1.0

Versions

Version 1.0.0 10.5281/zenodo.841984 Jan 7, 2018
Cite all versions? You can cite all versions by using the DOI 10.5281/zenodo.841983. This DOI represents all versions, and will always resolve to the latest one. Read more.

Share

Cite as

Export