Journal article Open Access

Continual learning for recurrent neural networks: An empirical evaluation

Andrea Cossu; Antonio Carta; Vincenzo Lomonaco; Davide Bacciu


Citation Style Language JSON Export

{
  "DOI": "10.1016/j.neunet.2021.07.021", 
  "container_title": "Neural Networks", 
  "language": "eng", 
  "title": "Continual learning for recurrent neural networks: An empirical evaluation", 
  "issued": {
    "date-parts": [
      [
        2021, 
        8, 
        5
      ]
    ]
  }, 
  "abstract": "<p>Learning continuously during all model lifetime is fundamental to deploy <a href=\"https://www.sciencedirect.com/topics/computer-science/machine-learning\">machine learning</a> solutions robust to drifts in the data distribution. Advances in Continual Learning (CL) with <a href=\"https://www.sciencedirect.com/topics/engineering/recurrent-neural-network\">recurrent neural networks</a> could pave the way to a large number of applications where incoming data is non stationary, like <a href=\"https://www.sciencedirect.com/topics/engineering/natural-language-processing\">natural language processing</a> and robotics. However, the existing body of work on the topic is still fragmented, with approaches which are application-specific and whose assessment is based on heterogeneous learning protocols and datasets. In this paper, we organize the literature on CL for sequential data processing by providing a categorization of the contributions and a review of the benchmarks. We propose two new benchmarks for CL with sequential data based on existing datasets, whose characteristics resemble real-world applications.</p>\n\n<p>We also provide a broad empirical evaluation of CL and Recurrent Neural Networks in class-incremental scenario, by testing their ability to mitigate forgetting with a number of different strategies which are not specific to sequential data processing. Our results highlight the key role played by the sequence length and the importance of a clear specification of the CL scenario.</p>", 
  "author": [
    {
      "family": "Andrea Cossu"
    }, 
    {
      "family": "Antonio Carta"
    }, 
    {
      "family": "Vincenzo Lomonaco"
    }, 
    {
      "family": "Davide Bacciu"
    }
  ], 
  "volume": "143", 
  "type": "article-journal", 
  "id": "5164245"
}
24
22
views
downloads
Views 24
Downloads 22
Data volume 34.9 MB
Unique views 22
Unique downloads 21

Share

Cite as