Kernel methods match Deep Neural Networks on TIMIT

By Po-Sen Huang, Haim Avron, Tara N. Sainath, Vikas Sindhwani, Bhuvana Ramabhadran. 2014

131 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Signal Processing

Citations in the corpus

Quasi-Monte Carlo Feature Maps for Shift-Invariant Kernels (2014, methodology): “Recent work on applying kernel methods to very large datasets, has shown their ability to achieve stateof-the-art accuracies that sometimes match those attainedby Deep Neural Networks (DNN) [Huang et al., 2014].”
Deep Fried Convnets (2014, background): “Our work is in line with recent interest in combining kernels with deep neural networks (Huang et al., 2014; Cho & Saul, 2009; Dai et al., 2014; Mairal et al., 2014).”
KeystoneML: Optimizing Pipelines for Large-Scale Advanced Analytics (2016, background): “Today’s advanced analytics applications increasingly use machine learning (ML) as a core technique in areas ranging from business intelligence to recommendation to natural language processing [39] and speech recognition [29].”
Generative Pretraining From Pixels (2020, background): “Other research cast doubt on the beneﬁts of deep unsupervised representations and re-ported strong results using a single layer of learned features (Coates et al., 2011), or even random features (Huang et al., 2014; May et al., 2017).”
On Deep Multi-View Representation Learning (2015, result): “For FKCCA, we ﬁnd it important to use a large number of random features M to get a competitive result, consistent with the ﬁndings of Huang et al. (2014) when using random Fourier features for speech data.”
How to Scale Up Kernel Methods to Be As Good As Deep Neural Nets (2014, methodology): “A recent work has employed the idea of randomly generated features in a speech recognition problem [22].”
Steps Toward Deep Kernel Methods from Infinite Neural Networks (2015, methodology): “Such estimators were recently used within kernel methods to match deep learning results in language processing [13, 10].”
A Survey of Modern Questions and Challenges in Feature Extraction (2015, background): “A successful attempt to solve that issue comes from the works that use distributed approximate feature maps (Lu et al., 2014; Huang et al., 2014), which allows them to build large scale kernel architectures that match deep networks in accuracy.”
From Dependence to Causation (2016, methodology): “For example, Huang et al. (2014) used 400.000 random features to build a state-of-the-art speech recognition system.”
An Exploration of Parameter Redundancy in Deep Networks with Circulant Projections (2015, unknown): “Our method is also related to the recent efforts of “shallow” neural networks, which show that sometimes shallow structures can match the performance of deep structures [28, 27, 19, 44].”
Random Laplace Feature Maps for Semigroup Kernels on Histograms (2014, methodology): “In recent years, approximations to kernel functions via explicit low-dimensional feature maps [18, 21, 15, 17, 12] have emerged as an appealing strategy to turn the complexity of learning nonlinear kernel methods back to that of training linear models, which typically scale linearly in the number of”
Ernest: Efficient Performance Prediction for Large-Scale Advanced Analytics (2016, methodology): “Figure 4: Execution DAG of a machine learning pipeline used for speech recognition [50].”
Automating model search for large scale machine learning (2015, background): “Baseline error for this modeling task is 95%, and state-of-the-art performance on this dataset is 35% error [25].”