Controllable Automatic Melody Composition Model across Pitch/Stress-accent Languages

This study proposes a model for automatically composing linguistically and musically natural song melodies reflecting the linguistic characteristics of both pitch-accent (e.g., Japanese) and stress-accent (e.g., English) languages as well as user's intentions. We have designed and provided publically, for more than 10 years, an automatic composition system (called ''Orpheus'') for Japanese lyrics. Extending the principle for lyrics written in stress-accent languages, a new compositional model was constructed by introducing a melodic rhythm generator formulated by a probabilistic model considering the relationship between stress of lyrics and rhythm intensity (linguistic naturalness and music theory) and the rhythm style chosen by the user (controllability). The parameters of the proposed model can be learnt from domain knowledge without large amounts of data. In our experimental evaluation, the proposed system achieved ratings equal to or better than state-of-the-art deep learning approaches in terms of musical coherence, singability and listenability.