Evaluation scheme for self-adaptive methods of coefficients of loss components of multi-objective loss function

This work addresses a challenge related to Multi-Objective Optimization in machine learning models training, specifically the problem of loss coefficients weight determination for physics grounded tasks. We propose a comprehensive comparative methodology for the analysis of balancing methods for loss function coefficients in deep learning models, to enhance replicability and comparisons across diverse applications, emphasizing the use of physical parameters as figures of merit. The proposed methodology is illustrated through the evaluation of self-adaptive methods for multicomponent loss coefficients in Graph Convolutional Neural Network (GCNN) models. The GCNN are trained to reproduce the forces acting on particles during coarse-grained molecular dynamics simulations. Criteria are outlined both for individual model assessment and for a statistical comparison between methods, highlight the differences in training-related characteristics, and performance metrics for the downstream task, across various self-balancing approaches.