Impact of High-Resource Speech Encoder Pretraining on Synthetic Data Requirements for Very-Low-Resource MLTR

Large language models (LLMs) have demonstrated potential in handling spoken inputs for high-resource languages, reaching state-of-the-art performance in various tasks. However, their applicability is still less explored in low-resource settings. This work investigates the use of Speech LLMs for low-resource Automatic Speech Recognition using the SLAM-ASR framework, where a trainable lightweight projector connects a speech encoder and a LLM. Firstly, we assess training data volume requirements to match Whisper-only performance, re-emphasizing the challenges of limited data. Secondly, we show th

Research goal: To what extent does pretraining the speech encoder on high-resource languages reduce the synthetic data volume required to reach 90% of peak MRR on very-low-resource MLTR tasks?

Autonomous synthesis report generated by Assignee Research. Tribunal consensus score: 7.5/10.