Conference paper Open Access

SkipW: Resource Adaptable RNN with Strict Upper Computational Limit

Mayet, Tsiry; Lambert, Anne; Le Guyadec, Pascal; Le Bolzer, Francoise; Schnitzler, Francois

We introduce Skip-Window, a method to allow recurrent neural networks (RNNs) to trade off accuracy for computational cost during the analysis of a sequence. Similarly to existing approaches, Skip-Window extends existing RNN cells by adding a mechanism to encourage the model to process fewer inputs. Unlike existing approaches, Skip-Window is able to respect a strict computational budget, making this model more suitable for limited hardware like edge devices. We evaluate this approach on four datasets: a human activity recognition task, sequential MNIST, IMDB and adding task. Our results show that Skip-Window is often able to exceed the accuracy of existing approaches for a lower computational cost while strictly limiting said cost.

Files (1.2 MB)
Name Size
skipw_resource_adaptable_rnn_with_strict_upper_computational_limit.pdf
md5:eba7c203e04072b0f6cb8638f0c17216
1.2 MB Download
17
11
views
downloads
All versions This version
Views 1717
Downloads 1111
Data volume 12.7 MB12.7 MB
Unique views 1414
Unique downloads 1010

Share

Cite as