Report Open Access

Testing the Plasticity of Reinforcement Learning Based Systems

Tonella, Paolo; Biagiola, Matteo

The data set available for pre-release training of a machine learning based system is often not representative of all possible execution contexts that the system will encounter in the field. Reinforcement Learning (RL) is a prominent approach among those that support continual learning, i.e., learning continually in the field, in the post-release phase. No study has so far investigated any method to test the plasticity of RL based systems, i.e., their capability to adapt to an execution context that may deviate from the training one. 

We propose an approach to test the plasticity of  RL based systems. The output of our approach is a quantification of the adaptation and anti-regression capabilities of the system, obtained by computing  the adaptation frontier of the system in a changed environment. We visualize such frontier as an adaptation/anti-regression heatmap in two dimensions, or as a clustered projection when more than two dimensions are involved. In this way, we provide developers with information on the amount of changes that can be accommodated by the continual learning component of the system, which is  key  to decide if online, in-the-field learning can be safely enabled or not.

Files (3.0 MB)
Name Size
TR-Precrime-2022-01.pdf
md5:1a453a18f1a4adc1dc3237d2383f9858
3.0 MB Download
85
127
views
downloads
All versions This version
Views 8554
Downloads 12753
Data volume 385.4 MB161.0 MB
Unique views 7347
Unique downloads 11646

Share

Cite as