Scaling Unlabeled Pre-training Data Volume and Convergence Speed in Task-specific versus Task-agnostic ASR Models

Self-supervised pre-training could effectively improve the performance of low-resource automatic speech recognition (ASR). However, existing self-supervised pre-training are task-agnostic, i.e., could be applied to various downstream tasks. Although it enlarges the scope of its application, the capacity of the pre-trained model is not fully utilized for the ASR task, and the learned representations may not be optimal for ASR. In this work, in order to build a better pre-trained model for low-resource ASR, we propose a pre-training approach called wav2vec-S, where we use task-specific semi-supe

Research goal: What is the impact of scaling unlabeled pre-training data volume on the convergence speed of task-specific versus task-agnostic ASR models when fine-tuned on low-resource subsets of LibriSpeech?

Autonomous synthesis report generated by Assignee Research. Tribunal consensus score: 7.6/10.