February 19, 2020 Dataset Open Access

Tahmasebi, Nina; Hengchen, Simon; Schlechtweg, Dominik; McGillivray, Barbara; Dubossarsky, Haim

Dublin Core Export

<?xml version='1.0' encoding='utf-8'?>
<oai_dc:dc xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/ http://www.openarchives.org/OAI/2.0/oai_dc.xsd">
  <dc:creator>Tahmasebi, Nina</dc:creator>
  <dc:creator>Hengchen, Simon</dc:creator>
  <dc:creator>Schlechtweg, Dominik</dc:creator>
  <dc:creator>McGillivray, Barbara</dc:creator>
  <dc:creator>Dubossarsky, Haim</dc:creator>
  <dc:date>2020-02-19</dc:date>
  <dc:description>This data collection contains the Swedish test data for SemEval 2020 Task 1: Unsupervised Lexical Semantic Change Detection:

- a Swedish text corpus pair (`corpus1/`, `corpus2/`)
- 31 lemmas which have been annotated for their lexical semantic change between the two corpora (`targets.txt`)

We sample from the KubHist2 corpus, digitized by the National Library of Sweden, and available through the Språkbanken corpus infrastructure Korp (Borin et al., 2012). The full corpus is available through a CC BY (attribution) license. Each word for which the lemmatizer in the Korp pipelien has found a lemma is replaced with the lemma. In cases where the lemmatizer cannot find a lemma, we leave the word as is (i.e., unlemmatized, no lower-casing). KubHist contains very frequent OCR errors, especially for the older data.More detail about the properties and quality of the Kubhist corpus can be found in (Adesam et al., 2019).

Lars Borin, Markus Forsberg, and Johan Roxendal. "Korp-the corpus infrastructure of Språkbanken." LREC. 2012.

Adesam, Yvonne, Dana Dannélls, and Nina Tahmasebi. "Exploring the Quality of the Digital Historical Newspaper Archive KubHist." DHN. 2019.

Corpus 1

- based on: Kubhist2
- language: Swedish
- time covered: 1790-1830
- size: ~71 million tokens
- format: lemmatized, sentence length &gt; 9 (before removal of punctuation), no punctuation, sentences randomly shuffled
- encoding: UTF-8
- note: contains frequent OCR errors

Corpus 2

- based on: Kubhist2
- language: Swedish
- time covered: 1895-1903
- size: ~111 million tokens
- format: lemmatized, sentence length &gt; 9 (before removal of punctuation), no punctuation, sentences randomly shuffled
- encoding: UTF-8
- note: contains OCR errors

Find more information on the data and SemEval2020 Task 1 in the paper referenced below.

Reference:

Dominik Schlechtweg, Barbara McGillivray, Simon Hengchen, Haim Dubossarsky and Nina Tahmasebi.SemEval 2020 Task 1: Unsupervised Lexical Semantic Change Detection. To appear in SemEval@COLING2020.</dc:description>
  <dc:description>The creation of the data was supported by the project Towards Computational Lexical Semantic Change Detection funded  by a project grant from the Swedish Research Council  (2019–2022;   dnr  2018-01184). 
It has also been created as part of the effort to construct and develop a Swedish national research infrastructure in support of research based on language data. This infrastructure -- Nationella språkbanken (the Swedish National Language Bank) -- is jointly funded for the period 2018--2024 by the Swedish Research Council (grant number 2017-00626) and its 10 partner institutions.</dc:description>
  <dc:identifier>https://zenodo.org/record/3672950</dc:identifier>
  <dc:identifier>10.5281/zenodo.3672950</dc:identifier>
  <dc:identifier>oai:zenodo.org:3672950</dc:identifier>
  <dc:language>swe</dc:language>
  <dc:relation>doi:10.5281/zenodo.3672949</dc:relation>
  <dc:relation>url:https://zenodo.org/communities/natural-language-processing</dc:relation>
  <dc:rights>info:eu-repo/semantics/openAccess</dc:rights>
  <dc:rights>https://creativecommons.org/licenses/by/2.0/legalcode</dc:rights>
  <dc:subject>unsupervised lexical semantic change detection, semantic change, SemEval2020, Kubhist2</dc:subject>
  <dc:title>Swedish Test Data for SemEval 2020 Task 1: Unsupervised Lexical Semantic Change Detection</dc:title>
  <dc:type>info:eu-repo/semantics/other</dc:type>
  <dc:type>dataset</dc:type>
</oai_dc:dc>