Stimulus Modality Matters: Impact of Perceptual Evaluations from Different Modalities on Speech Emotion Recognition System Performance

Initial public release of the code accompanying:

Huang-Cheng Chou et al., "Stimulus Modality Matters: Impact of Perceptual Evaluations from Different Modalities on Speech Emotion Recognition System Performance," IEEE ICASSP, 2025.

What this release contains

• Experiments quantifying how the stimulus modality shown to annotators (voice-only, face-only, audio-visual) propagates into downstream SER performance
• Fold definitions for each annotation condition, including the all-inclusive label set
• Evaluation pipeline built on s3prl

Links

• Paper: https://doi.org/10.1109/ICASSP49660.2025.10890558
• Preprint: https://arxiv.org/abs/2409.10762