Journal article Open Access

Continual learning for recurrent neural networks: An empirical evaluation

Andrea Cossu; Antonio Carta; Vincenzo Lomonaco; Davide Bacciu

JSON-LD (schema.org) Export

{
"inLanguage": {
"alternateName": "eng",
"@type": "Language",
"name": "English"
},
"description": "<p>Learning continuously during all model lifetime is fundamental to deploy <a href=\"https://www.sciencedirect.com/topics/computer-science/machine-learning\">machine learning</a> solutions robust to drifts in the data distribution. Advances in Continual Learning (CL) with <a href=\"https://www.sciencedirect.com/topics/engineering/recurrent-neural-network\">recurrent neural networks</a> could pave the way to a large number of applications where incoming data is non stationary, like <a href=\"https://www.sciencedirect.com/topics/engineering/natural-language-processing\">natural language processing</a> and robotics. However, the existing body of work on the topic is still fragmented, with approaches which are application-specific and whose assessment is based on heterogeneous learning protocols and datasets. In this paper, we organize the literature on CL for sequential data processing by providing a categorization of the contributions and a review of the benchmarks. We propose two new benchmarks for CL with sequential data based on existing datasets, whose characteristics resemble real-world applications.</p>\n\n<p>We also provide a broad empirical evaluation of CL and Recurrent Neural Networks in class-incremental scenario, by testing their ability to mitigate forgetting with a number of different strategies which are not specific to sequential data processing. Our results highlight the key role played by the sequence length and the importance of a clear specification of the CL scenario.</p>",
"creator": [
{
"affiliation": "University of Pisa",
"@type": "Person",
"name": "Andrea Cossu"
},
{
"affiliation": "University of Pisa",
"@type": "Person",
"name": "Antonio Carta"
},
{
"affiliation": "University of Pisa",
"@type": "Person",
"name": "Vincenzo Lomonaco"
},
{
"affiliation": "University of Pisa",
"@type": "Person",
"name": "Davide Bacciu"
}
],
"headline": "Continual learning for recurrent neural networks: An empirical evaluation",
"datePublished": "2021-08-05",
"url": "https://zenodo.org/record/5164245",
"keywords": [
"continual learning; recurrent neural networks"
],
"@context": "https://schema.org/",
"identifier": "https://doi.org/10.1016/j.neunet.2021.07.021",
"@id": "https://doi.org/10.1016/j.neunet.2021.07.021",
"@type": "ScholarlyArticle",
"name": "Continual learning for recurrent neural networks: An empirical evaluation"
}
24
22
views