Random Features for Large-Scale Kernel Machines

By Ali Rahimi, B. Recht. 2007

5,023 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Model Architectures

Citations in the corpus

Rethinking Attention with Performers (2020, background): “Conﬁgurations: h ( x ) = 1 , l = 2 , f 1 = sin , f 2 = cos correspond to shift-invariant kernels, in particular D = N (0 , I d ) leads to the Gaussian kernel K gauss (Rahimi & Recht, 2007).”
Generative Moment Matching Networks (2015, background): “Another possibility is to utilize random features (Rahimi & Recht, 2007).”
Deep double descent: where bigger models and more data hurt (2019, neutral): “In this section, for completeness sake, we show that both the model-and sample-wise double descent phenomena are not unique to deep neural networks—they exist even in the setting of Random Fourier Features of Rahimi & Recht (2008).”
When do neural networks outperform kernel methods? (2020, background): “Recent efforts to understand NN have put the spotlight on two linearizations of F NN, the random features [27] and the neural tangent [18] classes”
A Multi-View Embedding Space for Modeling Internet Images, Tags, and Their Semantics (2012, methodology): “For GIST features, we use the random Fourier feature mapping (Rahimi and Recht, 2007) that approximates the Gaussian kernel.”
Kernel Mean Embedding of Distributions: A Review and Beyonds (2016, methodology): “One of the most effective approaches depends on the random feature map (Rahimi and Recht 2007).”
Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness (2020, methodology): “Specifically, we first approximate the GP prior in (1) by deploying a low-rank approximation to the kernel matrix K = ΦΦ> using random features (Rahimi and Recht, 2008):”
Differentially Private Learning Needs Better Features (or Much More Data) (2020, critical): “Chaudhuri et al. (2011) propose to obtain private classifiers by approximating kernels using random features (Rahimi & Recht, 2008), but the very high dimensionality of the resulting learning problem makes it challenging to outperform our handcrafted features baseline.”
Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization (2016, methodology): “Features are randomly generated using the method described in Rahimi and Recht (2007) to approximate the RBF kernel, and these random features are then used as inputs to a ridge regression classiﬁer.”
Taking the Human Out of the Loop: A Review of Bayesian Optimization (2016, positive): “However, using recent spectral sampling techniques [20], [94], [125], we can draw an”
Invariant Causal Prediction for Nonlinear Models (2017, methodology): “In order to generalize the method to settings where an appropriate basis expansion of f is unknown, we look at ways to ﬁnd such a suitable basis expansion automatically by using random features (Williams and Seeger, 2001; Rahimi and Recht, 2008).”
Identifying Decision Points for Safe and Interpretable Reinforcement Learning in Hypotension Treatment (2021, background): “Rahimi and Recht (2008) showed that this process can be accelerated through the use of Random Fourier Features (RFF).”
Bayesian Learning of Kernel Embeddings (2016, methodology): “…to run cross-validation over a large space of hyperparameters. practice, when, e.g. large scale approximations based on random Fourier features (Rahimi and Recht, 2007) are used, we have not seen much attention paid to kernel learning – the features are often just one part of a complicated…”
A Kernel Test of Goodness of Fit (2016, methodology): “We next consider the random Fourier feature approximation to this model, where the log pdf, f , is approximated using a finite dictionary of random Fourier features (Rahimi & Recht, 2007).”