Deep Convolutional Networks vs. Transformers: Inference Efficiency and Memory Footprint on ImageNet

This report synthesises findings from 13 peer-reviewed papers addressing the following research question: How do the inference efficiency and memory footprint of deep convolutional neural networks compare to transformer-based architectures on ImageNet classification tasks, measured in terms of FLOPs and. While originally designed for natural language processing tasks, the self-attention mechanism has recently taken various computer vision areas by storm. However, the 2D nature of images brings three challenges for applying self-attention in computer vision: (1) treating images. 9 claims were extracted from source literature; 9 were independently verified against retrieved documents. An automated multi-reviewer quality assessment produced a score of 9.2/10. This report is a machine-generated literature synthesis and does not constitute original research.

Research goal: How do the inference efficiency and memory footprint of deep convolutional neural networks compare to transformer-based architectures on ImageNet classification tasks, measured in terms of FLOPs and latency on standardized hardware benchmarks?

Autonomous literature synthesis. Automated review score: 9.2/10. Full text and citation available at Assignee Research.