Unsupervised Singing Voice Conversion
By Eliya Nachmani, Lior Wolf. 2019
58 citations recorded by the source corpus.
We present a deep learning method for singing voice conversion. The proposed network is not conditioned on the text or on the notes, and it directly converts the audio of one singer to the voice of another. Training is performed without any form of supervision: no lyrics or any kind of phonetic features, no notes, and no matching samples between singers. The proposed network employs a single CNN encoder for all singers, a single WaveNet decoder, and a classifier that enforces the latent representation to be singer-agnostic. Each singer is represented by one embedding vector, which the decoder is conditioned on. In order to deal with relatively small datasets, we propose a new data augmentation scheme, as well as new training losses and protocols that are based on backtranslation. Our evaluation presents evidence that the conversion produces natural signing voices that are highly recognizable as the target singer.
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
Citations in the corpus
- Zero-Shot Voice Style Transfer with Only Autoencoder Loss (2019, background): “Atalla et al. (2019); Chou et al. (2018); Nachmani & Wolf (2019) conduct research on style transfer using autoencoder, but none has unveiled its distribution-matching property by properly designing the bottleneck.”
- Unsupervised Speech Decomposition via Triple Information Bottleneck (2020, background): “Besides, the time-domain deep generative model is gaining more research attention for voice conversion (Niwa et al., 2018; Nachmani & Wolf, 2019; Serrà et al., 2019).”
- Unsupervised Cross-Domain Singing Voice Conversion (2020, methodology): “For the single singer dataset, we use the same architecture as USVC but without the confusion term.”