Journal article Open Access

Memory Optimization Techniques in Neural Networks: A Review

Pratheeksha P; Pranav B M; Azra Nasreen

Sponsor(s)
Blue Eyes Intelligence Engineering and Sciences Publication (BEIESP)

Deep neural networks have been continuously evolving towards larger and more complex models to solve challenging problems in the field of AI. The primary bottleneck that restricts new network architectures is memory consumption. Running or training DNNs heavily relies on the hardware (CPUs, GPUs, or FPGA) which are either inadequate in terms of memory or hard-to-extend. This would further make it difficult to scale. In this paper, we review some of the latest memory footprint reduction techniques which would enable faster low model complexity. Additionally, it improves accuracy by increasing the batch size and developing wider and deeper neural networks with the same set of hardware resources. The paper emphasizes on memory optimization methods specific to CNN and RNN training.

Files (238.4 kB)
Name Size
F29910810621.pdf
md5:f3b5811d3d8d333a3e0edcf199ec3663
238.4 kB Download
16
17
views
downloads
Views 16
Downloads 17
Data volume 4.1 MB
Unique views 16
Unique downloads 17

Share

Cite as