Journal article Open Access

Continual learning for recurrent neural networks: An empirical evaluation

Andrea Cossu; Antonio Carta; Vincenzo Lomonaco; Davide Bacciu

Citation Style Language JSON Export

  "DOI": "10.1016/j.neunet.2021.07.021", 
  "container_title": "Neural Networks", 
  "language": "eng", 
  "title": "Continual learning for recurrent neural networks: An empirical evaluation", 
  "issued": {
    "date-parts": [
  "abstract": "<p>Learning continuously during all model lifetime is fundamental to deploy <a href=\"\">machine learning</a> solutions robust to drifts in the data distribution. Advances in Continual Learning (CL) with <a href=\"\">recurrent neural networks</a> could pave the way to a large number of applications where incoming data is non stationary, like <a href=\"\">natural language processing</a> and robotics. However, the existing body of work on the topic is still fragmented, with approaches which are application-specific and whose assessment is based on heterogeneous learning protocols and datasets. In this paper, we organize the literature on CL for sequential data processing by providing a categorization of the contributions and a review of the benchmarks. We propose two new benchmarks for CL with sequential data based on existing datasets, whose characteristics resemble real-world applications.</p>\n\n<p>We also provide a broad empirical evaluation of CL and Recurrent Neural Networks in class-incremental scenario, by testing their ability to mitigate forgetting with a number of different strategies which are not specific to sequential data processing. Our results highlight the key role played by the sequence length and the importance of a clear specification of the CL scenario.</p>", 
  "author": [
      "family": "Andrea Cossu"
      "family": "Antonio Carta"
      "family": "Vincenzo Lomonaco"
      "family": "Davide Bacciu"
  "volume": "143", 
  "type": "article-journal", 
  "id": "5164245"
Views 24
Downloads 22
Data volume 34.9 MB
Unique views 22
Unique downloads 21


Cite as