Planned intervention: On Wednesday April 3rd 05:30 UTC Zenodo will be unavailable for up to 2-10 minutes to perform a storage cluster upgrade.
Published September 26, 2022 | Version v1
Conference paper Open

Pruned Lightweight Encoders for Computer Vision

  • 1. Tampere University

Description

Latency-critical computer vision systems, such as autonomous driving or drone control, require fast image or video compression when offloading neural network inference to a remote computer. To ensure low latency on a near-sensor edge device, we propose the use of lightweight encoders with constant bitrate and pruned encoding configurations, namely, ASTC and JPEG XS. Pruning introduces significant distortion which we show can be recovered by retraining the neural network with compressed data after decompression. Such an approach does not modify the network architecture or require coding format modifications. By retraining with compressed datasets, we reduced the classification accuracy and segmentation mean intersection over union (mIoU) degradation due to ASTC compression to 4.9–5.0 percentage points (pp) and 4.4–4.0 pp, respectively. With the same method, the mIoU lost due to JPEG XS compression at the main profile was restored to 2.7–2.3 pp. In terms of encoding speed, our ASTC encoder implementation is 2.3x faster than JPEG. Even though the JPEG XS reference encoder requires optimizations to reach low latency, we showed that disabling significance flag coding saves 22–23% of encoding time at the cost of 0.4–0.3 mIoU after retraining.

Files

Pruned_Lightweight_Encoders_for_Computer_Vision__MMSP_2022_.pdf

Files (314.5 kB)

Additional details

Funding

CPSoSaware – Cross-layer cognitive optimization tools & methods for the lifecycle support of dependable CPSoS 871738
European Commission