Deep Speaker: an End-to-End Neural Speaker Embedding System

By Chao Li, Xiaokong Ma, Bing Jiang, Xiangang Li, Xuewei Zhang, Xiao Liu, Ying Cao, Ajay Kannan, Zhenyao Zhu. 2017

533 citations recorded by the source corpus.

We present Deep Speaker, a neural speaker embedding system that maps utterances to a hypersphere where speaker similarity is measured by cosine similarity. The embeddings generated by Deep Speaker can be used for many tasks, including speaker identification, verification, and clustering. We experiment with ResCNN and GRU architectures to extract the acoustic features, then mean pool to produce utterance-level speaker embeddings, and train using triplet loss based on cosine similarity. Experiments on three distinct datasets suggest that Deep Speaker outperforms a DNN-based i-vector baseline. For example, Deep Speaker reduces the verification equal error rate by 50% (relatively) and improves the identification accuracy by 60% (relatively) on a text-independent dataset. We also present results that suggest adapting from a model trained with Mandarin can improve accuracy for English speaker recognition.

Subfields: Artificial Intelligence, Representation & Self-Supervised Learning, Speech & Audio, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Representation & Self-Supervised Learning

In Speech & Audio

In Signal Processing

Citations in the corpus

AdaSpeech: Adaptive Text to Speech for Custom Voice (2021, background): “Some works only ﬁne-tune the speaker embedding (Arik et al., 2018; Chen et al., 2018), or train a speaker encoder module (Arik et al., 2018; Jia et al., 2018; Cooper et al., 2020; Li et al., 2017; Wan et al., 2018) that does not need ﬁne-tuning during adaptation.”
Exploring the Encoding Layer and Loss Function in End-to-End Speaker and Language Recognition System (2018, methodology): “ears, with the merit of large labeled datasets, enormous computation capability, and effective network architectures, emerging progress towards end-to-end learning opens up a new area for exploration [19, 20, 21, 22, 23]. In our previous works [24, 25], we proposed a learnable dictionary encoding (L”
Neural Voice Cloning with a Few Samples (2018, background): “processing Speaker embedding is a well-established approach to encode discriminative information in speakers. It has been used in many speech processing tasks such as speaker recognition/veriﬁcation (Li et al., 2017), speaker diarization (Rouvier et al., 2015), automatic speech recognition (Doddipat”
Generalized End-to-End Loss for Speaker Verification (2017, background): “In recent years, more efforts have been focusing on using neural networks for speaker verification, while the most successful systems use end-to-end training [8, 9, 10, 11, 12].”
Deep Voice 2: Multi-Speaker Neural Text-to-Speech (2017, methodology): “Speaker classiﬁcation has been studied with both traditional GMM-based methods (e.g., Reynolds et al., 2000) and more recently with deep learning approaches (e.g., Li et al., 2017).”
Adaspeech 2: Adaptive Text to Speech with Untranscribed Data (2021, background): “TTS adaptation pipeline [1, 2, 3, 4, 5, 6, 7, 8] usually finetunes a well-trained multi-speaker TTS model [9, 10] (source model) with few adaptation data, and achieves good similarity with ground-truth human voice and high voice quality.”
From Speaker Verification to Multispeaker Speech Synthesis, Deep Transfer with Feedback Constraint (2020, methodology): “In the speaker verification field, deep speaker feature learning systems [15, 16] proposed in recent years have achieved comparable performance or even surpassed the classical ivector systems [17].”
BOFFIN TTS: Few-Shot Speaker Adaptation by Bayesian Optimization (2020, methodology): “Many approaches use pre-trained auxiliary encoding networks to extract speaker characteristics to be combined with linguistic features as inputs to a TTS model [7, 8, 9, 10].”
Cross-Lingual, Multi-Speaker Text-To-Speech Synthesis Using Neural Speaker Embedding (2019, methodology): “As mentioned in [18], the Figure 2: Speaker encoder.”
Cross-Lingual Speaker Adaptation Using Domain Adaptation and Speaker Consistency Loss for Text-To-Speech Synthesis (2021, methodology): “The only difference here is that we use a more powerful model called ResCNN [13] as the speaker en-coder’s implementation instead of simple long short-term memory to encode the mel-spectrograms to cope with the complexity of multilingual data.”
Learning Speaker Representations with Mutual Information (2018, methodology): “1 reports the sentence-level classification error rates achieved with binary cross-entropy (BCE), MINE, Noise Constructive Estimation (NCE), and the triplet loss used in [30].”
STYLER: Style Factor Modeling with Rapidity and Robustness via Speech Decomposition for Expressive and Controllable Neural Text to Speech (2021, methodology): “In this work, Deep Speaker [22] is employed since it is faithful to embed speaker identities without additional features such as noise robustness.”
Voxceleb: Large-scale speaker verification in the wild (2020, methodology): “In some cases, the network is further trained for verification using the contrastive loss (Nagrani et al., 2017; Chung et al., 2018; Chen et al., 2011) or other metric learning losses such as the triplet loss (Li et al., 2017).”
Investigation of Fast and Efficient Methods for Multi-Speaker Modeling and Speaker Adaptation (2021, methodology): “As for average model design, most of previous works create the multi-speaker (so call “average”) models based on Tacotron or its varieties models [14, 15].”
Teacher-Student Training for Text-Independent Speaker Recognition (2018, background): “End-to-end training has been investigated in a number of studies, where SR systems were trained on distance-based objectives, such as pairwise cosine distance [10, 11], triplet loss [12, 13] and its generalised version [14].”
Learning Word Embeddings: Unsupervised Methods for Fixed-size Representations of Variable-length Speech Segments (2018, methodology): “DNNs have been used to derive embeddings by training them with a supervised objective, although not for speech recognition per se: [20, 21] alternate convolutional and pooling layers, finishing with a fixed length vector.”
Disentangled Speaker and Language Representations Using Mutual Information Minimization and Domain Adaptation for Cross-Lingual TTS (2021, methodology): “Similarly to the conventional method [13], we used ResCNN [25] as the implementation of the feature extractor.”