T L DEEPIKA ROY, NULAKA SRINIVASU

Multimodal emotion recognition Multimodal physiological and behavioral emotion recognition is of critical importance in affective computing, human-computer interaction (HCI), and mental-health analytics. However, current deep learning models generally do not take modalities into account (disregarding their causal and temporal inter-dependencies) and ignore personality-based variability that is essential to realistic affect modelling. To address these shortcomings, the given paper proposes CausalGraph-EmotionNet, a personality-conscious causal graph transformer, which combines Neural-ODE-based temporal evolution with causal attention-assisted multimodal fusion. The AFFECT data of each modality (EEG, electrodermal activity, facial activity, eye gaze, pupil dilation, and cursor movement) is modeled as a dynamic causal graph the time-varying connectivity of which reflects time-varying functional and directional interactions. The merged embeddings are optimized by personality-conditioned causal attention systems, which allows making individualized and interpretable inferences about emotions. Large-scale experiments on the AFFET dataset indicate that CausalGraph-EmotionNet has 84.6% accuracy and 80.8% macro-F1, outperforming CNN, RNN, GCN, Transformer and PhysioGraph-Transformer. The model significantly enhances the identification of more complex affective conditions like fear and disgust, it is resilient to a 40% loss in modality and has interpretable causal maps that bridge personality dimensions and modality salience. The findings make CausalGraph-EmotionNet a state-of-the-art, explainable and causally motivated architecture of multimodal emotion recognition - a unification of data-driven learning with psychologically relevant causal inferences.