Conference paper Open Access

Structured Pruning of LSTMs via Eigenanalysis and Geometric Median for Mobile Multimedia and Deep Learning Applications

Gkalelis, Nikolaos; Mezaris, Vasileios

In this paper, a novel structured pruning approach for learning efficient long short-term memory (LSTM) network architectures is proposed. More specifically, the eigenvalues of the covariance matrix associated with the responses of each LSTM layer are computed and utilized to quantify the layers’ redundancy and automatically obtain an individual pruning rate for each layer. Subsequently, a Geometric Median based (GM-based) criterion is used to identify and prune in a structured way the most redundant LSTM units, realizing the pruning rates derived in the previous step. The experimental evaluation on the Penn Treebank text corpus and the large-scale YouTube-8M audio-video dataset for the tasks of word-level prediction and visual concept detection, respectively, shows the efficacy of the proposed approach. Source code is made publicly available at: https://github.com/bmezaris/lstm_structured_pruning_geometric_median

Files (253.0 kB)
Name Size
ism2020_preprint.pdf
md5:09f01069fd210ff9f0114eb054d79f93
253.0 kB Download
112
45
views
downloads
Views 112
Downloads 45
Data volume 11.4 MB
Unique views 111
Unique downloads 44

Share

Cite as