February 9, 2022 Report Open Access

Tonella, Paolo; Biagiola, Matteo

DataCite XML Export

<?xml version='1.0' encoding='utf-8'?>
<resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://datacite.org/schema/kernel-4" xsi:schemaLocation="http://datacite.org/schema/kernel-4 http://schema.datacite.org/meta/kernel-4.1/metadata.xsd">
  <identifier identifierType="DOI">10.5281/zenodo.6355271</identifier>
  <creators>
    <creator>
      <creatorName>Tonella, Paolo</creatorName>
      <givenName>Paolo</givenName>
      <familyName>Tonella</familyName>
      <affiliation>Università della Svizzera italiana</affiliation>
    </creator>
    <creator>
      <creatorName>Biagiola, Matteo</creatorName>
      <givenName>Matteo</givenName>
      <familyName>Biagiola</familyName>
      <affiliation>Università della Svizzera italiana</affiliation>
    </creator>
  </creators>
  <titles>
    <title>Testing the Plasticity of Reinforcement Learning Based Systems</title>
  </titles>
  <publisher>Zenodo</publisher>
  <publicationYear>2022</publicationYear>
  <dates>
    <date dateType="Issued">2022-02-09</date>
  </dates>
  <resourceType resourceTypeGeneral="Report"/>
  <alternateIdentifiers>
    <alternateIdentifier alternateIdentifierType="url">https://zenodo.org/record/6355271</alternateIdentifier>
  </alternateIdentifiers>
  <relatedIdentifiers>
    <relatedIdentifier relatedIdentifierType="DOI" relationType="IsVersionOf">10.5281/zenodo.6026648</relatedIdentifier>
  </relatedIdentifiers>
  <rightsList>
    <rights rightsURI="https://creativecommons.org/licenses/by/4.0/legalcode">Creative Commons Attribution 4.0 International</rights>
    <rights rightsURI="info:eu-repo/semantics/openAccess">Open Access</rights>
  </rightsList>
  <descriptions>
    <description descriptionType="Abstract">&lt;p&gt;The data set available for pre-release training of a machine learning based system is often not representative of all possible execution contexts that the system will encounter in the field. Reinforcement Learning (RL) is a prominent approach among those that support continual learning, i.e., learning continually in the field, in the post-release phase. No study has so far investigated any method to test the plasticity of RL based systems, i.e., their capability to adapt to an execution context that may deviate from the training one.&amp;nbsp;&lt;/p&gt;

&lt;p&gt;We propose an approach to test the plasticity of&amp;nbsp; RL based systems. The output of our approach is a quantification of the adaptation and anti-regression capabilities of the system, obtained by computing&amp;nbsp; the adaptation frontier of the system in a changed environment. We visualize such frontier as an adaptation/anti-regression heatmap in two dimensions, or as a clustered projection when more than two dimensions are involved. In this way, we provide developers with information on the amount of changes that can be accommodated by the continual learning component of the system, which is&amp;nbsp; key&amp;nbsp; to decide if online, in-the-field learning can be safely enabled or not.&lt;/p&gt;</description>
  </descriptions>
  <fundingReferences>
    <fundingReference>
      <funderName>European Commission</funderName>
      <funderIdentifier funderIdentifierType="Crossref Funder ID">10.13039/100010661</funderIdentifier>
      <awardNumber awardURI="info:eu-repo/grantAgreement/EC/H2020/787703/">787703</awardNumber>
      <awardTitle>Self-assessment Oracles for Anticipatory Testing</awardTitle>
    </fundingReference>
  </fundingReferences>
</resource>