There is a newer version of the record available.

Published February 9, 2022 | Version v1
Report Open

Testing the Plasticity of Reinforcement Learning Based Systems

  • 1. Università della Svizzera italiana

Description

The data set available for pre-release training of a machine learning based system is often not representative of all possible execution contexts that the system will encounter in the field. Reinforcement Learning (RL) is a prominent approach among those that support continual learning, i.e., learning continually in the field, in the post-release phase. No study has so far investigated any method to test the plasticity of RL based systems, i.e., their capability to adapt to an execution context that may deviate from the training one. 

We propose an approach to test the plasticity of  RL based systems. The output of our approach is a quantification of the adaptation and anti-regression capabilities of the system, obtained by computing  the adaptation frontier of the system in a changed environment. We visualize such frontier as an adaptation/anti-regression heatmap in two dimensions, or as a clustered projection when more than two dimensions are involved. In this way, we provide developers with information on the amount of changes that can be accommodated by the continual learning component of the system, which is  key  to decide if online, in-the-field learning can be safely enabled or not.

Files

TR-Precrime-2022-01.pdf

Files (3.0 MB)

Name Size Download all
md5:86d268817978c0b9d6b03490d02164a5
3.0 MB Preview Download

Additional details

Funding

PRECRIME – Self-assessment Oracles for Anticipatory Testing 787703
European Commission