Cluster adaptive training for deep neural network

By Tian Tan, Yanmin Qian, Maofan Yin, Yimeng Zhuang, Kai Yu. 2015

75 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jinyu Li
2. Dong Yu
3. Tara N. Sainath
4. Bo Li
5. Khe Chai Sim
6. Li Deng
7. Arun Narayanan
8. Yifan Gong

In Signal Processing

1. Jinyu Li
2. Dong Yu
3. Khe Chai Sim
4. Tara N. Sainath
5. Bo Li
6. Li Deng
7. Arun Narayanan
8. Yifan Gong

Citations in the corpus

• Salvaging Federated Learning by Local Adaptation (2020, background): “Many techniques are not compatible with federated learning because they require an ensemble of models (Tan et al., 2015) or are speech-specific (Miao et al., 2015), but (Huang et al., 2015) connects personalization and multi-task learning.”
• Personalization of End-to-End Speech Recognition on Mobile Devices for Named Entities (2019, background): “There are existing speaker adaptation techniques for neural networks that offer a more compact model representation by having a large number of global parameters shared across different speakers and a smaller number of speaker-specific parameters [13, 14, 15, 16, 17, 18, 19, 20].”
• Domain Adaptation Using Factorized Hidden Layer for Robust Automatic Speech Recognition (2018, background): “…have been previously studied, including i-vector feature augmentation [19, 20], speaker-code adaptation [21], cluster adaptive training (CAT) [22, 23], factorized hidden layer [9], learning hidden unit contribution (LHUC) [24], singular value decomposition based subspace adaptation [25] and…”
• An Investigation of Deep Neural Networks for Multilingual Speech Recognition Training and Adaptation (2017, methodology): “Cluster adaptive training (CAT) was extended from Gaussian mixture model (GMM) to DNN [11].”
• An Investigation Into On-device Personalization of End-to-end Automatic Speech Recognition Models (2019, unknown): “A common solution to this problem is to adapt a well-trained model using a small amount of user-specific data to build a personalized model for each user [4, 5, 6, 7, 8, 9, 10, 11, 12, 13].”
• Recent progresses in deep learning based acoustic models (2017, background): “Therefore, the number of bases in CAT is usually constrained to few [113], [114].”
• Stimulated Deep Neural Network for Speech Recognition (2016, methodology): “In the DNN-CAT [11, 12] and the multi-basis adaptive neural network [13] models, speaker-dependent interpolation weights are introduced to combine different DNN modules to handle the acoustic distortions.”
• Factorized Hidden Layer Adaptation for Deep Neural Network Based Acoustic Modeling (2016, methodology): “The FHL formulation is similar to Cluster Adaptive Training (CAT) [22], [23] for DNN, but uses a relatively larger number of rank-1 bases and the speaker parameters are bootstrapped using i-vectors.”