Dataset Open Access

SemEval-2020 Task 1: Unsupervised Lexical Semantic Change Detection

Schlechtweg, Dominik; McGillivray, Barbara; Hengchen, Simon; Dubossarsky, Haim; Tahmasebi, Nina


JSON-LD (schema.org) Export

{
  "inLanguage": {
    "alternateName": "eng", 
    "@type": "Language", 
    "name": "English"
  }, 
  "description": "<p><strong>Authors</strong></p>\n\n<p>Dominik Schlechtweg, Barbara McGillivray, Simon Hengchen, Haim Dubossarsky, and Nina Tahmasebi</p>\n\n<p><strong>Description</strong></p>\n\n<p>This data collection contains the <strong>post-evaluation</strong> data for <a href=\"https://languagechange.org/semeval\">SemEval-2020 Task 1: Unsupervised Lexical Semantic Change Detection</a>:</p>\n\n<ul>\n\t<li>the starting kit to download data, and examples for competing in the CodaLab challenge including baselines</li>\n\t<li>the true binary change scores of the targets for Subtask 1, and their true graded change scores for Subtask 2 (<code>test_data_truth/</code>),</li>\n\t<li>the scoring program used to score submissions against the true test data in the evaluation and post-evaluation phase (<code>scoring_program/</code>),</li>\n\t<li>the results of the evaluation phase including\n\t<ul>\n\t\t<li>the final rankings of the participating teams by their best submission (<code>results/rankings_teams.csv</code>),</li>\n\t\t<li>the submitted files of each team (<code>results/submissions/</code>),</li>\n\t\t<li>an overview of the results for each submission ordered by team (<code>results/submissions_results.csv</code>),</li>\n\t\t<li>analysis plots (<code>plots/</code>) displaying the results:\n\t\t<ul>\n\t\t\t<li>under <code>per_target/</code> we provide the gold change scores and the normalized prediction error of target words plotted against their frequency and polysemy statistics,</li>\n\t\t\t<li>under <code>per_team/</code> we provide the model predictions from the best submission per team (per subtask) plotted against frequency/polysemy statistics and performance on gold data (gray lines give the correlation with the respective variable in the gold data); we also provide plots of visualizing the teams&#39; prediction similarities.</li>\n\t\t</ul>\n\t\t</li>\n\t</ul>\n\t</li>\n</ul>\n\n<p>Some remarks:</p>\n\n<ul>\n\t<li>the paper referenced below remains the only source for the rankings between teams,</li>\n\t<li>some teams were disqualified, and are thus removed from the analyses and the rankings present in the paper,</li>\n\t<li>some teams have changed names, resulting in a discrepancy between team names under <code>results/</code> and team names in the paper. The paper contains a key to match old names with new names.</li>\n</ul>\n\n<p><strong>Test Data </strong>for SemEval-2020 Task 1: Unsupervised Lexical Semantic Change Detection can be found using the links below:</p>\n\n<ul>\n\t<li><a href=\"https://www.ims.uni-stuttgart.de/en/research/resources/corpora/sem-eval-ulscd-eng/\">English</a></li>\n\t<li><a href=\"https://www.ims.uni-stuttgart.de/en/research/resources/corpora/sem-eval-ulscd-ger/\">German</a></li>\n\t<li><a href=\"https://zenodo.org/record/3734089\">Latin</a></li>\n\t<li><a href=\"https://zenodo.org/record/3730550\">Swedish</a></li>\n</ul>\n\n<p>Please find more information on the provided data in the paper referenced below.</p>\n\n<p><strong>Reference</strong></p>\n\n<p>Dominik Schlechtweg, Barbara McGillivray, Simon Hengchen, Haim Dubossarsky and Nina Tahmasebi. 2020. <a href=\"https://languagechange.org/semeval\">SemEval-2020 Task 1: Unsupervised Lexical Semantic Change Detection</a>. SemEval@COLING2020.</p>\n\n<p>The resources are freely available for education, research and other non-commercial purposes.</p>\n\n<pre><code>@inproceedings{schlechtweg2020semeval,\ntitle = \"{S}em{E}val-2020 {T}ask 1: {U}nsupervised {L}exical {S}emantic {C}hange {D}etection\",\nauthor = \"Schlechtweg, Dominik and McGillivray, Barbara and Hengchen, Simon and Dubossarsky, Haim and Tahmasebi, Nina\",\nbooktitle = \"To appear in Proceedings of the 14th International Workshop on Semantic Evaluation\",\nyear = \"2020\",\naddress = \"Barcelona, Spain\",\npublisher = \"Association for Computational Linguistics\"}</code></pre>\n\n<p>&nbsp;</p>", 
  "license": "https://creativecommons.org/licenses/by/4.0/legalcode", 
  "creator": [
    {
      "affiliation": "IMS, University of Stuttgart", 
      "@id": "https://orcid.org/0000-0002-0685-2576", 
      "@type": "Person", 
      "name": "Schlechtweg, Dominik"
    }, 
    {
      "affiliation": "The Alan Turing Institute and University of Cambridge", 
      "@id": "https://orcid.org/0000-0003-3426-8200", 
      "@type": "Person", 
      "name": "McGillivray, Barbara"
    }, 
    {
      "affiliation": "Spr\u00e5kbanken, University of Gothenburg", 
      "@id": "https://orcid.org/0000-0002-8453-7221", 
      "@type": "Person", 
      "name": "Hengchen, Simon"
    }, 
    {
      "affiliation": "University of Cambridge", 
      "@id": "https://orcid.org/0000-0002-2818-6113", 
      "@type": "Person", 
      "name": "Dubossarsky, Haim"
    }, 
    {
      "affiliation": "Spr\u00e5kbanken, University of Gothenburg", 
      "@id": "https://orcid.org/0000-0003-1688-1845", 
      "@type": "Person", 
      "name": "Tahmasebi, Nina"
    }
  ], 
  "url": "https://zenodo.org/record/3931969", 
  "datePublished": "2020-05-27", 
  "version": "v1", 
  "@type": "Dataset", 
  "keywords": [
    "unsupervised lexical semantic change detection", 
    "semantic change", 
    "SemEval2020 Task1", 
    "English", 
    "German", 
    "Latin", 
    "Swedish"
  ], 
  "@context": "https://schema.org/", 
  "distribution": [
    {
      "contentUrl": "https://zenodo.org/api/files/c1a8bcba-e620-461d-9d2a-523aac2bf4b7/semeval2020_ulscd_posteval.zip", 
      "encodingFormat": "zip", 
      "@type": "DataDownload"
    }
  ], 
  "identifier": "https://doi.org/10.5281/zenodo.3931969", 
  "@id": "https://doi.org/10.5281/zenodo.3931969", 
  "workFeatured": {
    "url": "http://alt.qcri.org/semeval2020/", 
    "alternateName": "SemEval-2020", 
    "location": "Barcelona, Spain", 
    "@type": "Event", 
    "name": "Proceedings of the 14th International Workshop on Semantic Evaluation"
  }, 
  "name": "SemEval-2020 Task 1: Unsupervised Lexical Semantic Change Detection"
}
839
235
views
downloads
All versions This version
Views 839628
Downloads 235229
Data volume 995.9 MB970.5 MB
Unique views 746573
Unique downloads 230224

Share

Cite as