VRAM OPTIMIZATION FOR TRANSFORMERS BASED NUTRITION PREDICTION MODEL

Accurate estimation of nutritional content from food images can promote better dietary awareness and health management. Recent advances in deep learning have enabled transformer-based models to achieve state-of-the-art results for nutrition prediction; however, their high video memory (VRAM) requirements limit accessibility and scalability. In this work, we propose the integration of four VRAM optimization techniques into an existing state-of-the-art transformer-based nutrition prediction framework to reduce memory consumption during training. The optimization methods include Automatic Mixed Precision (AMP), gradient checkpointing, gradient accumulation, and the Adam8bit optimizer. The framework employs a Swin Transformer (Swin-T) backbone with a Feature Pyramid Network (FPN) and a Swin-TUNA segmentation module for food-region detection. Experiments on the Nutrition5k dataset demonstrate a 77% reduction in peak VRAM and a 78% reduction in reserved VRAM during training, while maintaining model accuracy (PMAE = 16.9%, improved from 17.2%). Although training time increased due to re-computation and accumulation overhead, the proposed optimization approach enables stable training of large transformer architectures on mid-range GPUs. This work provides a practical direction for developing memory-efficient nutrition prediction systems, improving the accessibility of transformer-based models for future research and deployment.