Journal article Open Access

# Continual learning for recurrent neural networks: An empirical evaluation

Andrea Cossu; Antonio Carta; Vincenzo Lomonaco; Davide Bacciu

### Citation Style Language JSON Export

{
"DOI": "10.1016/j.neunet.2021.07.021",
"container_title": "Neural Networks",
"language": "eng",
"title": "Continual learning for recurrent neural networks: An empirical evaluation",
"issued": {
"date-parts": [
[
2021,
8,
5
]
]
},
"abstract": "<p>Learning continuously during all model lifetime is fundamental to deploy <a href=\"https://www.sciencedirect.com/topics/computer-science/machine-learning\">machine learning</a> solutions robust to drifts in the data distribution. Advances in Continual Learning (CL) with <a href=\"https://www.sciencedirect.com/topics/engineering/recurrent-neural-network\">recurrent neural networks</a> could pave the way to a large number of applications where incoming data is non stationary, like <a href=\"https://www.sciencedirect.com/topics/engineering/natural-language-processing\">natural language processing</a> and robotics. However, the existing body of work on the topic is still fragmented, with approaches which are application-specific and whose assessment is based on heterogeneous learning protocols and datasets. In this paper, we organize the literature on CL for sequential data processing by providing a categorization of the contributions and a review of the benchmarks. We propose two new benchmarks for CL with sequential data based on existing datasets, whose characteristics resemble real-world applications.</p>\n\n<p>We also provide a broad empirical evaluation of CL and Recurrent Neural Networks in class-incremental scenario, by testing their ability to mitigate forgetting with a number of different strategies which are not specific to sequential data processing. Our results highlight the key role played by the sequence length and the importance of a clear specification of the CL scenario.</p>",
"author": [
{
"family": "Andrea Cossu"
},
{
"family": "Antonio Carta"
},
{
"family": "Vincenzo Lomonaco"
},
{
"family": "Davide Bacciu"
}
],
"volume": "143",
"type": "article-journal",
"id": "5164245"
}
24
22
views