Fast Function to Function Regression
By Junier B. Oliva, Willie Neiswanger, Barnab'as P'oczos, Eric P. Xing, Hy Trac, Shirley Ho, Jeff Schneider. 2014
31 citations recorded by the source corpus.
We analyze the problem of regression when both input covariates and output responses are functions from a nonparametric function class. Function to function regression (FFR) covers a large range of interesting applications including timeseries prediction problems, and also more general tasks like studying a mapping between two separate types of distributions. However, previous nonparametric estimators for FFR type problems scale badly computationally with the number of input/output pairs in a data-set. Given the complexity of a mapping between general functions it may be necessary to consider large datasets in order to achieve a low estimation risk. To address this issue, we develop a novel scalable nonparametric estimator, the Triple-Basis Estimator (3BE), which is capable of operating over data-sets with many instances. To the best of our knowledge, the 3BE is the first nonparametric FFR estimator that can scale to massive data-sets. We analyze the 3BE’s risk and derive an upperbound rate. Furthermore, we show an improvement of several orders of magnitude in terms of prediction speed and a reduction in error over previous estimators in various real-world datasets.
Subfields: Artificial Intelligence, Control and Systems Engineering
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Noah A. Smith
- Dani Yogatama
- Roy Schwartz
- Yejin Choi
- Hao Peng
- Lingpeng Kong
- Graham Neubig
- Nikolaos Pappas
In Control and Systems Engineering
- Noah A. Smith
- Zoltán Szabó
- Dani Yogatama
- Bharath K. Sriperumbudur
- Arthur Gretton
- Kenji Fukumizu
- Dino Sejdinovic
Citations in the corpus
- Optimal Rates for Random Fourier Features (2015, background): “We also propose a RFF approximation to derivatives of a kernel with a theoretical study on its approximation quality.”
- Random Feature Attention (2021, methodology): “Random feature methods proved successful in speeding up kernel methods (Oliva et al., 2015; Avron et al., 2017; Sun, 2019, inter alia ), and more recently are used to efficiently approximate soft-max (Rawat et al., 2019).”