February 19, 2020 Dataset Open Access

Tahmasebi, Nina; Hengchen, Simon; Schlechtweg, Dominik; McGillivray, Barbara; Dubossarsky, Haim

Citation Style Language JSON Export

{
  "publisher": "Zenodo", 
  "DOI": "10.5281/zenodo.3672950", 
  "language": "swe", 
  "title": "Swedish Test Data for SemEval 2020 Task 1: Unsupervised Lexical Semantic Change Detection", 
  "issued": {
    "date-parts": [
      [
        2020, 
        2, 
        19
      ]
    ]
  }, 
  "abstract": "<p>This data collection contains the Swedish test data for <a href=\"https://languagechange.org/semeval\">SemEval 2020 Task 1: Unsupervised Lexical Semantic Change Detection:</a></p>\n\n<p>- a Swedish text corpus pair (`corpus1/`, `corpus2/`)<br>\n- 31 lemmas which have been annotated for their lexical semantic change between the two corpora (`targets.txt`)</p>\n\n<p>We sample from the KubHist2 corpus, digitized by the National Library of Sweden, and available through the Spr&aring;kbanken corpus infrastructure Korp (<a href=\"https://www.researchgate.net/profile/Markus_Forsberg/publication/266352576_Korp_-_the_corpus_infrastructure_of_Sprakbanken/links/55bf1ee008aed621de121ba3/Korp-the-corpus-infrastructure-of-Sprakbanken.pdf\">Borin et al., 2012</a>). The full corpus is available through a CC BY (attribution) license. Each word for which the lemmatizer in the Korp pipelien has found a lemma is replaced with the lemma. In cases where the lemmatizer cannot find a lemma, we leave the word as is (i.e., unlemmatized, no lower-casing). KubHist contains very frequent OCR errors, especially for the older data.More detail about the properties and quality of the Kubhist corpus can be found in (<a href=\"https://www.diva-portal.org/smash/get/diva2:1358014/FULLTEXT01.pdf#page=28\">Adesam et al., 2019</a>).</p>\n\n<p>Lars Borin, Markus Forsberg, and Johan Roxendal. &quot;Korp-the corpus infrastructure of Spr&aring;kbanken.&quot; <em>LREC</em>. 2012.</p>\n\n<p>Adesam, Yvonne, Dana Dann&eacute;lls, and Nina Tahmasebi. &quot;Exploring the Quality of the Digital Historical Newspaper Archive KubHist.&quot; <em>DHN</em>. 2019.</p>\n\n<p>Corpus 1</p>\n\n<p>- based on: <a href=\"https://spraakbanken.gu.se/korp/?mode=kubhist\">Kubhist2</a><br>\n- language: Swedish<br>\n- time covered: 1790-1830<br>\n- size: ~71 million tokens<br>\n- format: lemmatized, sentence length &gt; 9 (before removal of punctuation), no punctuation, sentences randomly shuffled<br>\n- encoding: UTF-8<br>\n- note: contains frequent OCR errors</p>\n\n<p>Corpus 2</p>\n\n<p>- based on:&nbsp;<a href=\"https://spraakbanken.gu.se/korp/?mode=kubhist\">Kubhist2</a><br>\n- language: Swedish<br>\n- time covered: 1895-1903<br>\n- size: ~111 million tokens<br>\n- format: lemmatized, sentence length &gt; 9 (before removal of punctuation), no punctuation, sentences randomly shuffled<br>\n- encoding: UTF-8<br>\n- note: contains OCR errors</p>\n\n<p>Find more information on the data and SemEval2020 Task 1 in the paper referenced below.</p>\n\n<p>Reference:</p>\n\n<p>Dominik Schlechtweg, Barbara McGillivray, Simon Hengchen, Haim Dubossarsky and Nina Tahmasebi.<a href=\"https://competitions.codalab.org/competitions/20948\">SemEval 2020 Task 1: Unsupervised Lexical Semantic Change Detection</a>. To appear in SemEval@COLING2020.</p>", 
  "author": [
    {
      "family": "Tahmasebi, Nina"
    }, 
    {
      "family": "Hengchen, Simon"
    }, 
    {
      "family": "Schlechtweg, Dominik"
    }, 
    {
      "family": "McGillivray, Barbara"
    }, 
    {
      "family": "Dubossarsky, Haim"
    }
  ], 
  "note": "The creation of the data was supported by the project Towards Computational Lexical Semantic Change Detection funded  by a project grant from the Swedish Research Council  (2019\u20132022;   dnr  2018-01184). \nIt has also been created as part of the effort to construct and develop a Swedish national research infrastructure in support of research based on language data. This infrastructure -- Nationella spr\u00e5kbanken (the Swedish National Language Bank) -- is jointly funded for the period 2018--2024 by the Swedish Research Council (grant number 2017-00626) and its 10 partner institutions.", 
  "version": "v1", 
  "type": "dataset", 
  "id": "3672950"
}