Published August 13, 2021 | Version v1
Conference paper Open

Improving the Accuracy of Early Exits in Multi-Exit Architectures via Curriculum Learning

  • 1. DIGIT, Department of Electrical and Computer Engineering, Aarhus University, Denmark

Description

Deploying deep learning services for time-sensitive and resource-constrained settings such as IoT using edge computing systems is a challenging task that requires dynamic adjustment of inference time. Multi-exit architectures allow deep neural networks to terminate their execution early in order to adhere to tight deadlines at the cost of accuracy. To mitigate this cost, in this paper we introduce a novel method called Multi-Exit Curriculum Learning that utilizes curriculum learning, a training strategy for neural networks that imitates human learning by sorting the training samples based on their difficulty and gradually introducing them to the network. Experiments on CIFAR-10 and CIFAR-100 datasets and various configurations of multi-exit architectures show that our method consistently improves the accuracy of early exits compared to the standard training approach.

Notes

This work was partly funded by the European Union's Horizon 2020 research and innovation programme under grant agreement No 957337, and by the Danish Council for Independent Research under Grant No. 9131-00119B. This publication reflects the authors views only. The European Commission and the Danish Council for Independent Research are not responsible for any use that may be made of the information it contains.

Files

IJCNN2021_AU.pdf

Files (2.8 MB)

Name Size Download all
md5:e61fac8945325d215a55720c34683cf6
2.8 MB Preview Download

Additional details

Related works

Is published in
Conference paper: 10.1109/IJCNN52387.2021.9533875 (DOI)
Is supplemented by
Software: https://gitlab.au.dk/maleci/MultiExitCurriculumLearning (URL)

Funding

MARVEL – Multimodal Extreme Scale Data Analytics for Smart Cities Environments 957337
European Commission