Conference paper Open Access

Some like it tough: Improving model generalization via progressively increasing the training difficulty

Fassold, Hannes

MARC21 XML Export

<?xml version='1.0' encoding='UTF-8'?>
<record xmlns="">
  <controlfield tag="005">20211026134844.0</controlfield>
  <controlfield tag="001">5596848</controlfield>
  <datafield tag="856" ind1="4" ind2=" ">
    <subfield code="s">349241</subfield>
    <subfield code="z">md5:da0eca6acff8e644aa86bdfaff378840</subfield>
    <subfield code="u"></subfield>
  <datafield tag="542" ind1=" " ind2=" ">
    <subfield code="l">open</subfield>
  <datafield tag="260" ind1=" " ind2=" ">
    <subfield code="c">2021-10-25</subfield>
  <datafield tag="909" ind1="C" ind2="O">
    <subfield code="p">openaire</subfield>
    <subfield code="p">user-ai4media</subfield>
    <subfield code="o"></subfield>
  <datafield tag="100" ind1=" " ind2=" ">
    <subfield code="u">JOANNEUM RESEARCH</subfield>
    <subfield code="a">Fassold, Hannes</subfield>
  <datafield tag="245" ind1=" " ind2=" ">
    <subfield code="a">Some like it tough: Improving model generalization via progressively increasing the training difficulty</subfield>
  <datafield tag="980" ind1=" " ind2=" ">
    <subfield code="a">user-ai4media</subfield>
  <datafield tag="536" ind1=" " ind2=" ">
    <subfield code="c">951911</subfield>
    <subfield code="a">A European Excellence Centre for Media, Society and Democracy</subfield>
  <datafield tag="540" ind1=" " ind2=" ">
    <subfield code="u"></subfield>
    <subfield code="a">Creative Commons Attribution 4.0 International</subfield>
  <datafield tag="650" ind1="1" ind2="7">
    <subfield code="a">cc-by</subfield>
    <subfield code="2"></subfield>
  <datafield tag="520" ind1=" " ind2=" ">
    <subfield code="a">&lt;p&gt;In this work, we propose to progressively increase the training difficulty during learning a neural network model via a novel strategy which we call mini-batch trimming. This strategy makes sure that the optimizer puts its focus in the later training stages on the more difficult samples, which we identify as the ones with the highest loss in the current mini-batch. The strategy is very easy to integrate into an existing training pipeline and does not necessitate a change of the network model. Experiments on several image classification problems show that mini-batch trimming is able to increase the generalization ability (measured via final test error) of the trained model.&lt;/p&gt;</subfield>
  <datafield tag="773" ind1=" " ind2=" ">
    <subfield code="n">doi</subfield>
    <subfield code="i">isVersionOf</subfield>
    <subfield code="a">10.5281/zenodo.5596847</subfield>
  <datafield tag="024" ind1=" " ind2=" ">
    <subfield code="a">10.5281/zenodo.5596848</subfield>
    <subfield code="2">doi</subfield>
  <datafield tag="980" ind1=" " ind2=" ">
    <subfield code="a">publication</subfield>
    <subfield code="b">conferencepaper</subfield>
All versions This version
Views 2121
Downloads 2020
Data volume 7.0 MB7.0 MB
Unique views 1919
Unique downloads 1919


Cite as