Neural Network Language Modeling with Letter-Based Features and Importance Sampling

By Hainan Xu, Ke Li, Yiming Wang, Jian Wang, Shiyin Kang, Xie Chen, Daniel Povey, Sanjeev Khudanpur. 2018

72 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Large Language Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yu Zhang
2. Jinyu Li
3. Yonghui Wu
4. Quoc V. Le
5. Chung-Cheng Chiu
6. Shinji Watanabe
7. Ruoming Pang
8. Gabriel Synnaeve

In Large Language Models

1. Quoc V. Le
2. Daniel Povey
3. Yu Zhang
4. Yonghui Wu
5. Wei Han
6. Chung-Cheng Chiu
7. Yannick Estève
8. Hainan Xu

Citations in the corpus

• Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition (2020, background): “The papers corresponding to each data point have been collected in the bibliography [6, 8, 9, 13–21].”
• TED-LIUM 3: twice as much data and corpus repartition for experiments on speaker adaptation (2018, methodology): “We also considered a RNNLM with letter-based features and importance sampling [15], coupled with a pruned approach to lattice-rescoring [14].”
• A Pruned Rnnlm Lattice-Rescoring Algorithm for Automatic Speech Recognition (2018, methodology): “The RNNLMs are trained with TensorFlow [4], using the objective function described in [18] which prevents the need to compute a normalization term during test time scenarios.”
• Semi-Orthogonal Low-Rank Matrix Factorization for Deep Neural Networks (2018, methodology): “The first-pass decoding uses a 3-gram backoff LM which we rescore with an RNNLM [15, 16], with explicit modeling for silence probabilities [17].”