Planned intervention: On Wednesday April 3rd 05:30 UTC Zenodo will be unavailable for up to 2-10 minutes to perform a storage cluster upgrade.
Published January 24, 2023 | Version v1
Conference paper Open

Batch-efficient EigenDecomposition for Small and Medium Matrices

  • 1. UNITN

Description

EigenDecomposition (ED) is at the heart of many computer
vision algorithms and applications. One crucial bottleneck limiting its
usage is the expensive computation cost, particularly for a mini-batch
of matrices in the deep neural networks. In this paper, we propose a
QR-based ED method dedicated to the application scenarios of computer
vision. Our proposed method performs the ED entirely by batched
matrix/vector multiplication, which processes all the matrices simultaneously
and thus fully utilizes the power of GPUs. Our technique is based
on the explicit QR iterations by Givens rotation with double Wilkinson
shifts. With several acceleration techniques, the time complexity of QR
iterations is reduced from O(n5) to O(n3). The numerical test shows that
for small and medium batched matrices (e.g., dim<32) our method can
be much faster than the Pytorch SVD function. Experimental results on
visual recognition and image generation demonstrate that our methods
also achieve competitive performances.

Files

136830566 (2).pdf

Files (9.2 MB)

Name Size Download all
md5:fcbcbde1bca82a2d3e4455128e97676b
9.2 MB Preview Download

Additional details

Funding

AI4Media – A European Excellence Centre for Media, Society and Democracy 951911
European Commission