Rectified linear neural networks with tied-scalar regularization for LVCSR

By Shiliang Zhang, Hui Jiang, Si Wei, Lirong Dai. 2015

12 citations recorded by the source corpus.

Subfields: Aerospace Engineering, Computer Networks and Communications, Control and Systems Engineering, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

1. Kai Chen
2. Diederik P. Kingma
3. Qiang Huo
4. Tim Salimans
5. Shiliang Zhang

In Computer Networks and Communications

1. Kai Chen
2. Diederik P. Kingma
3. Qiang Huo
4. Tim Salimans
5. Lirong Dai

In Control and Systems Engineering

1. Kai Chen
2. Diederik P. Kingma
3. Qiang Huo
4. Tim Salimans

In Natural Language Processing

1. Kai Chen
2. Diederik P. Kingma
3. Qiang Huo
4. Tim Salimans
5. Ming Lei
6. Shiliang Zhang
7. Lirong Dai
8. Zhijie Yan

Citations in the corpus

• Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks (2016, methodology): “[31, 33]) but earlier work typically still performed optimization in the w-parameterization, only applying the normalization after each step of stochastic gradient descent.”
• Deep-FSMN for Large Vocabulary Continuous Speech Recognition (2018, methodology): “For the hybrid DNN-HMM baseline system, we follow the same training procedure as described in [29] to train the conventional context dependent DNN-HMM using the tied-state alignment obtained from a MLE trained GMM-HMM baseline system.”
• Scalable training of deep learning machines by incremental block training with intra-block parallel optimization and blockwise model-update filtering (2016, methodology): “Yet a large mini-batch SGD method with tied scalar regularization is proposed in [33] to train DNNs with rectified linear units (ReLUs) and achieves promising results.”