ExpressiveSinger: Multilingual and Multi-Style Score-based Singing Voice Synthesis with Expressive Performance Control


Shuqi Dai, Ming-Yu Liu, Rafael Valle, Siddharth Gururani


In Proceedings of the 32nd ACM International Conference on Multimedia (MM ’24), October 28 - November 1, 2024, Melbourne, VIC, Australia. https://doi.org/10.1145/3664647.3681642


Demo 1: Comparison With Existing Works and Human Singing (in Chinese)

Input Lyrics Ours(ExpressiveSinger)
22kHz, Score Timing
GT Human Singing
22kHz
DiffSinger
24kHz, GT MIDI Timing
VISinger2
22kHz, GT MIDI Timing
热恋的时刻最任性
力气请告诉我
这最后的出口在爱过了才有
而暂停算不算放弃
你送我的眼泪让他留在雨天
这世界的太阳因为自信才能把我照亮
这舞台的中央有我才闪亮
竹篱上停留着蜻蜓
青春嫩绿得
很鲜明
风铃摇晃曾经
而我听见下雨的声音
我付出一直很小心
就怕情绪红了眼睛
的脸上透明
而我听见下雨的声音
想起你用唇语说爱情
终于听见下雨的声音
无聊变得有话聊有变化了
我永远爱你到老
无聊变得有话聊有变化了
我放慢了步调感觉像是喝醉了
一辈子暖暖的好
我永远爱你到老
不要再沉默徘徊
冲破这层层阻碍
走吧没人能让你停下来
我想我早开始感受着释怀
我才明白外面世界如此精彩
时间飞这生命似钟摆
是你让我对世界充满着热爱

Demo 2: Different Style, Language, Techniques

Generated (ExpressiveSinger) GT Human Singing

Demo 3: Zero-shot Senarios

A Happy Birthday Song in Chinese sung by different singers/styles. This song is not in the training data and is generated from the score from scratch. Many singers in this demo have never sung Chinese in the training data (zero-shot senario):