Robustness Enhancement in SLAM-ASR Models via High-Resource Speech Data Augmentation

Large language models (LLMs) have demonstrated potential in handling spoken inputs for high-resource languages, reaching state-of-the-art performance in various tasks. However, their applicability is still less explored in low-resource settings. This work investigates the use of Speech LLMs for low-resource Automatic Speech Recognition using the SLAM-ASR framework, where a trainable lightweight projector connects a speech encoder and a LLM. Firstly, we assess training data volume requirements to match Whisper-only performance, re-emphasizing the challenges of limited data. Secondly, we show th

Research goal: Does increasing the volume of high-resource speech data during multilingual pretraining improve the robustness of SLAM-ASR models against noise in low-resource language inputs?

Autonomous synthesis report generated by Assignee Research. Tribunal consensus score: 7.5/10.