Towards a More Refined Training Process for Neural Networks: Applying Layer-wise Relevance Propagation to Understand and Improve Classification Performance on Imbalanced Datasets

In this thesis, we consider the problem of neural network (NN) training on imbalanced datasets. The generalization performance of state-of-the-art Deep Neural Networks heavily relies on the dataset composition, and class imbalance can lead to undesirable effects such as overfitting on majority classes or disregard of minority classes. Furthermore, imbalanced datasets are not a rare case, especially when the data is taken from a real-world setting. To mitigate their negative effects on state-ofthe-art networks, the training behavior of a neural network on imbalanced data is analyzed and adapted with the goal of achieving a more balanced performance, i.e. with the aim to obtain an improved generalization. For this purpose, heatmaps generated via Layer-wise Relevance Propagation (LRP) (Bach et al., 2015) are employed as a measure of (class-wise) understanding. As an example case, the extremely popular and widely used VGG-16 model (Simonyan and Zisserman, 2014) is chosen as the network architecture and the Adience benchmark (Eidinger, Enbar, and Hassner, 2014) as an imbalanced real-world dataset. We start with a short overview over the methods relevant for this thesis, i.e., established approaches for dealing with imbalanced datasets when training neural networks and explainability methods, especially LRP. In a first experiment, the usage of LRP-heatmaps as a measure of understanding is then validated by determining their correlation to traditional evaluation metrics, e.g., test accuracy and F1-score. For this purpose, secondary metrics are derived from the LRP-heatmaps, to enable the comparison to the traditional evaluation metrics. A strong connection between the LRP-heatmap-based metrics and the traditional evaluation metrics is uncovered here. Moreover, we find that the LRP-heatmap-based metrics are indicative of how well the classes separate w.r.t. the prediction function f(x). These properties motivate the second experiment, where the LRP-heatmaps and the secondary metrics derived from them are applied practically for making class-wise adaptations to the NNs’ training behavior. Here, the expressivity of LRP-heatmaps is demonstrated for this purpose by achieving more balanced class-wise performances with training-improving methods based on LRP, since, e.g., the derived methods force the neural network model during training to immediately consider previously neglected classes.