Scaling of Inference Latency with Decoder Depth in Neural Source-Filter Models for Real-Time Symbolic-to-Audio Synthesis

Speech synthesis and music audio generation from symbolic input differ in many aspects but share some similarities. In this study, we investigate how text-to-speech synthesis techniques can be used for piano MIDI-to-audio synthesis tasks. Our investigation includes Tacotron and neural source-filter waveform models as the basic components, with which we build MIDI-to-audio synthesis systems in similar ways to TTS frameworks. We also include reference systems using conventional sound modeling techniques such as sample-based and physical-modeling-based methods. The subjective experimental results

Research goal: How does the inference latency of neural source-filter models scale with increased decoder depth in real-time symbolic-to-audio synthesis?

Autonomous synthesis report generated by SOVEREIGN Research Kernel. Tribunal consensus score: 7.7/10.