Journal article Open Access

Continual learning for recurrent neural networks: An empirical evaluation

Andrea Cossu; Antonio Carta; Vincenzo Lomonaco; Davide Bacciu


JSON Export

{
  "files": [
    {
      "links": {
        "self": "https://zenodo.org/api/files/a738e08a-4fd7-450e-ac9f-56ce338bcef2/2103.07492.pdf"
      }, 
      "checksum": "md5:e168aef7dad0e2f8f6d5d8711223b168", 
      "bucket": "a738e08a-4fd7-450e-ac9f-56ce338bcef2", 
      "key": "2103.07492.pdf", 
      "type": "pdf", 
      "size": 1584864
    }
  ], 
  "owners": [
    246602
  ], 
  "doi": "10.1016/j.neunet.2021.07.021", 
  "stats": {
    "version_unique_downloads": 21.0, 
    "unique_views": 22.0, 
    "views": 24.0, 
    "version_views": 24.0, 
    "unique_downloads": 21.0, 
    "version_unique_views": 22.0, 
    "volume": 34867008.0, 
    "version_downloads": 22.0, 
    "downloads": 22.0, 
    "version_volume": 34867008.0
  }, 
  "links": {
    "doi": "https://doi.org/10.1016/j.neunet.2021.07.021", 
    "latest_html": "https://zenodo.org/record/5164245", 
    "bucket": "https://zenodo.org/api/files/a738e08a-4fd7-450e-ac9f-56ce338bcef2", 
    "badge": "https://zenodo.org/badge/doi/10.1016/j.neunet.2021.07.021.svg", 
    "html": "https://zenodo.org/record/5164245", 
    "latest": "https://zenodo.org/api/records/5164245"
  }, 
  "created": "2021-08-05T17:29:58.811519+00:00", 
  "updated": "2021-08-10T08:47:28.267518+00:00", 
  "conceptrecid": "5164244", 
  "revision": 3, 
  "id": 5164245, 
  "metadata": {
    "access_right_category": "success", 
    "doi": "10.1016/j.neunet.2021.07.021", 
    "description": "<p>Learning continuously during all model lifetime is fundamental to deploy <a href=\"https://www.sciencedirect.com/topics/computer-science/machine-learning\">machine learning</a> solutions robust to drifts in the data distribution. Advances in Continual Learning (CL) with <a href=\"https://www.sciencedirect.com/topics/engineering/recurrent-neural-network\">recurrent neural networks</a> could pave the way to a large number of applications where incoming data is non stationary, like <a href=\"https://www.sciencedirect.com/topics/engineering/natural-language-processing\">natural language processing</a> and robotics. However, the existing body of work on the topic is still fragmented, with approaches which are application-specific and whose assessment is based on heterogeneous learning protocols and datasets. In this paper, we organize the literature on CL for sequential data processing by providing a categorization of the contributions and a review of the benchmarks. We propose two new benchmarks for CL with sequential data based on existing datasets, whose characteristics resemble real-world applications.</p>\n\n<p>We also provide a broad empirical evaluation of CL and Recurrent Neural Networks in class-incremental scenario, by testing their ability to mitigate forgetting with a number of different strategies which are not specific to sequential data processing. Our results highlight the key role played by the sequence length and the importance of a clear specification of the CL scenario.</p>", 
    "language": "eng", 
    "title": "Continual learning for recurrent neural networks: An empirical evaluation", 
    "license": {
      "id": "CC-BY-4.0"
    }, 
    "journal": {
      "volume": "143", 
      "title": "Neural Networks"
    }, 
    "relations": {
      "version": [
        {
          "count": 1, 
          "index": 0, 
          "parent": {
            "pid_type": "recid", 
            "pid_value": "5164244"
          }, 
          "is_last": true, 
          "last_child": {
            "pid_type": "recid", 
            "pid_value": "5164245"
          }
        }
      ]
    }, 
    "communities": [
      {
        "id": "teaching-h2020"
      }
    ], 
    "grants": [
      {
        "code": "871385", 
        "links": {
          "self": "https://zenodo.org/api/grants/10.13039/501100000780::871385"
        }, 
        "title": "A computing toolkit for building efficient autonomous applications leveraging humanistic intelligence", 
        "acronym": "TEACHING", 
        "program": "Horizon 2020 Framework Programme - Research and Innovation action", 
        "funder": {
          "doi": "10.13039/501100000780", 
          "acronyms": [], 
          "name": "European Commission", 
          "links": {
            "self": "https://zenodo.org/api/funders/10.13039/501100000780"
          }
        }
      }
    ], 
    "keywords": [
      "continual learning; recurrent neural networks"
    ], 
    "publication_date": "2021-08-05", 
    "creators": [
      {
        "affiliation": "University of Pisa", 
        "name": "Andrea Cossu"
      }, 
      {
        "affiliation": "University of Pisa", 
        "name": "Antonio Carta"
      }, 
      {
        "affiliation": "University of Pisa", 
        "name": "Vincenzo Lomonaco"
      }, 
      {
        "affiliation": "University of Pisa", 
        "name": "Davide Bacciu"
      }
    ], 
    "access_right": "open", 
    "resource_type": {
      "subtype": "article", 
      "type": "publication", 
      "title": "Journal article"
    }
  }
}
24
22
views
downloads
Views 24
Downloads 22
Data volume 34.9 MB
Unique views 22
Unique downloads 21

Share

Cite as