MS-RGT-LF-XAI: Multi-Scale Radiomic–Graph–Transformer Late-Fusion Network with Explainable Artificial Intelligence for Breast Cancer Histopathology Classification

Deep learning-based histopathological classification of breast tissue into normal, benign, in situ carcinoma and invasive carcinoma remains challenging because of multi-scale feature representations, high sensitivity to hyperparameter selection and poor interpretability. To address these challenges, we propose MS-RGT-LF-XAI, an explainable multi-branch late-fusion framework that integrates multi-scale convolutional neural network (CNN) features with handcrafted radiomic descriptors, a nuclei-level graph convolutional network (GCN) that models cellular relationships, and a transformer encoder that learns contextual representations. The embeddings of the independent branches are concatenated for the final classification; no cross-attention is used between GCN nodes and transformer tokens. The hyperparameters of MS-RGT-LF-XAI were tuned with the Bluefin Trevally Optimizer (BTO), an existing
2
algorithm that is applied here and is not introduced as part of this work. BTO was compared with particle swarm optimisation (PSO), grey wolf optimisation (GWO), the whale optimisation algorithm (WOA), differential evolution (DE), the covariance matrix adaptation evolution strategy (CMA-ES), Optuna, Bayesian optimisation (BO) and random search under equal evaluation budgets. Model behaviour was examined post hoc with SHAP (SHapley Additive exPlanations) and attention rollout. The framework was evaluated retrospectively on the Breast Cancer Histology (BACH) dataset of 400 images, and all experiments were repeated across ten random seeds. MS-RGT-LF-XAI obtained a macro-averaged accuracy of 97.84%, an F1-score of 97.62%, an area under the receiver operating characteristic curve (AUROC) of 0.989 and a Matthews correlation coefficient (MCC) of 0.970. These results indicate promising classification performance on the examined dataset. However, the small test partition (about 60 images) limits the precision of the performance estimates and the ability to establish statistically significant differences among closely competing models. Because the evaluation was retrospective and limited to a single dataset, generalisability beyond this dataset and clinical utility were not demonstrated. Future work will therefore include prospective multicentre studies with pathologist reader evaluations.