5890998
doi
10.5281/zenodo.5890998
oai:zenodo.org:5890998
Jauhiainen, Heidi
University of Helsinki
HeLI-OTS 1.2 with createmodels.java
Jauhiainen, Tommi
University of Helsinki
info:eu-repo/semantics/openAccess
Creative Commons Attribution 4.0 International
https://creativecommons.org/licenses/by/4.0/legalcode
language identification
<p>HeLI off-the-shelf language identifier with language models for 200 languages.</p>
<p>Usage:<br>
java -jar HeLI.jar -r <infile> -w <outfile></p>
<p>The program will read the <infile> and classify the language of each line as one of the 200 languages it knows<br>
and writes the results, one ISO 639-3 code per line, into file <outfile>.</p>
<p>You can use the -c option to make the program print a confidence score for the identification after each language code.</p>
<p>Usage:<br>
java -jar HeLI.jar -c -r <infile> -w <outfile></p>
<p>You can give the list of comma-separated ISO 639-3 identifiers for relevant languages after -l option.</p>
<p>Usage:<br>
java -jar HeLI.jar -r <infile> -w <outfile> -l fin,swe,eng</p>
<p>You can give the number of top-scored languages to print after the -t option. (overrides confidence)</p>
<p>Usage:<br>
java -jar HeLI.jar -r <infile> -w <outfile> -l fin,swe,eng -t 2</p>
<p>If you omit both of the filenames, the program will read the standard input one line at a time and write the result to standard output.</p>
<p>It can identify c. 3000 sentences per second using one core on a 2021 laptop and around 3 gigabytes of memory.</p>
<p>If you use this program in producing scientific publications, please refer to: <br>
@inproceedings{jauhiainen-etal-2017-evaluation,<br>
title = "Evaluation of language identification methods using 285 languages",<br>
author = "Jauhiainen, Tommi and<br>
Lind{\'e}n, Krister and<br>
Jauhiainen, Heidi",<br>
booktitle = "Proceedings of the 21st Nordic Conference on Computational Linguistics",<br>
month = may,<br>
year = "2017",<br>
address = "Gothenburg, Sweden",<br>
publisher = "Association for Computational Linguistics",<br>
url = "https://www.aclweb.org/anthology/W17-0221",<br>
pages = "183--191",<br>
}</p>
<p>Producing and publishing this software has been partly supported by The Finnish Research Impact Foundation Tandem Industry Academia -funding in cooperation with Lingsoft.</p>
Zenodo
2022-01-22
info:eu-repo/semantics/other
4780897
1.2
1662803859.726529
11419
md5:bb0ae3b700049fd806e2a043e01265d6
https://zenodo.org/records/5890998/files/LICENSE
745
md5:5d551dcb80653aaac5ecebae98842826
https://zenodo.org/records/5890998/files/supporting_functions.py
1003
md5:fa3de39cf2e93085759e3f97cb9f4d0f
https://zenodo.org/records/5890998/files/run_HeLI.py
44132999
md5:efd3371472a6b3a93133773a6c09d87b
https://zenodo.org/records/5890998/files/LanguageModels.zip
13674
md5:95657280ee492a6ab4844eeb4454a5c0
https://zenodo.org/records/5890998/files/HeLI.class
6672
md5:6812296c1a1397e778b674e66327feaf
https://zenodo.org/records/5890998/files/createmodels.java
2734
md5:e6f06930e25726624e53eb7a901e0874
https://zenodo.org/records/5890998/files/README.md
22452
md5:c71b0f3cd044bf424908d905fa0a7a97
https://zenodo.org/records/5890998/files/HeLI.java
39
md5:bb91c0c41fd40f3fb8a7c4f98c9a7c87
https://zenodo.org/records/5890998/files/HeLI.mf
44050741
md5:8537531e1e6f74f67a58fcfc0ac302e3
https://zenodo.org/records/5890998/files/HeLI.jar
884
md5:f44bcfe8a8a8108095b6bc35cea8e31d
https://zenodo.org/records/5890998/files/languagelist
public
10.5281/zenodo.4780897
isVersionOf
doi