Dataset Open Access

Temporally-Informed Analysis of Named Entity Recognition

Rijhwani, Shruti; Preoțiuc-Pietro, Daniel

JSON Export

  "files": [
      "links": {
        "self": ""
      "checksum": "md5:ba79cfa2ec554a7bc40241f86b344280", 
      "bucket": "a2fe4f96-2590-4c4f-a549-6b701221d940", 
      "key": "", 
      "type": "zip", 
      "size": 185283
  "owners": [
  "doi": "10.5281/zenodo.3899040", 
  "stats": {
    "version_unique_downloads": 73.0, 
    "unique_views": 266.0, 
    "views": 287.0, 
    "version_views": 287.0, 
    "unique_downloads": 73.0, 
    "version_unique_views": 266.0, 
    "volume": 13896225.0, 
    "version_downloads": 75.0, 
    "downloads": 75.0, 
    "version_volume": 13896225.0
  "links": {
    "doi": "", 
    "conceptdoi": "", 
    "bucket": "", 
    "conceptbadge": "", 
    "html": "", 
    "latest_html": "", 
    "badge": "", 
    "latest": ""
  "conceptdoi": "10.5281/zenodo.3899039", 
  "created": "2020-06-17T16:56:52.272923+00:00", 
  "updated": "2020-06-17T22:18:22.795583+00:00", 
  "conceptrecid": "3899039", 
  "revision": 2, 
  "id": 3899040, 
  "metadata": {
    "access_right_category": "success", 
    "doi": "10.5281/zenodo.3899040", 
    "description": "<p>This repository contains the data set developed for the paper:</p>\n\n<p>&ldquo;Shruti Rijhwani and Daniel Preo\u021biuc-Pietro. <em>Temporally-Informed Analysis of Named Entity Recognition.</em> In Proceedings of the Association for Computational Linguistics (ACL). 2020.&rdquo;</p>\n\n<p>It includes 12,000 tweets annotated for the named entity recognition task. The tweets are uniformly distributed over the years 2014-2019, with 2,000 tweets from each year. The goal is to have a temporally diverse corpus to account for data drift over time when building NER models.</p>\n\n<p>The entity types annotated are locations (LOC), persons (PER) and organizations (ORG). The tweets are preprocessed to replace usernames and URLs with a unique token. Hashtags are left intact and can be annotated as named entities.</p>\n\n<p><strong>Format</strong></p>\n\n<p>The repository contains the annotations in JSON format.</p>\n\n<p>Each year-wise file has the tweet IDs along with token-level annotations. The Public Twitter Search API (<a href=\"\"></a>) can be used extract the text for the tweet corresponding to the tweet IDs.</p>\n\n<p><strong>Data Splits</strong></p>\n\n<p>Typically, NER models are trained and evaluated on annotations available at the model building time, but are used to make predictions on data from a future time period. This setup makes the model susceptible to temporal data drift, leading to lower performance on future data as compared to the test set.</p>\n\n<p>To examine this effect, we use tweets from the years 2014-2018 as the training set and random splits of the 2019 tweets as the development and test sets. These splits simulate the scenario of making predictions on data from a future time period.</p>\n\n<p>The development and test splits are provided in the JSON format.</p>\n\n<p><strong>Use</strong></p>\n\n<p>Please cite the data set and the accompanying paper if you found the resources in this repository useful.</p>", 
    "language": "eng", 
    "title": "Temporally-Informed Analysis of Named Entity Recognition", 
    "license": {
      "id": "CC-BY-4.0"
    "relations": {
      "version": [
          "count": 1, 
          "index": 0, 
          "parent": {
            "pid_type": "recid", 
            "pid_value": "3899039"
          "is_last": true, 
          "last_child": {
            "pid_type": "recid", 
            "pid_value": "3899040"
    "keywords": [
      "named entity recognition", 
      "twitter ner", 
      "temporal analysis", 
      "information extraction"
    "publication_date": "2020-06-17", 
    "creators": [
        "affiliation": "Bloomberg", 
        "name": "Rijhwani, Shruti"
        "affiliation": "Bloomberg", 
        "name": "Preo\u021biuc-Pietro, Daniel"
    "meeting": {
      "acronym": "ACL2020", 
      "url": "", 
      "dates": "5-10 July 2020", 
      "title": "The 58th Annual Meeting of the Association for Computational Linguistics"
    "access_right": "open", 
    "resource_type": {
      "type": "dataset", 
      "title": "Dataset"
    "related_identifiers": [
        "scheme": "doi", 
        "identifier": "10.5281/zenodo.3899039", 
        "relation": "isVersionOf"
All versions This version
Views 287287
Downloads 7575
Data volume 13.9 MB13.9 MB
Unique views 266266
Unique downloads 7373


Cite as