The Random Forest Kernel and other kernels for big data from random partitions
By Alex Davies, Zoubin Ghahramani. 2014
72 citations recorded by the source corpus.
We present Random Partition Kernels, a new class of kernels derived by demonstrating a natural connection between random partitions of objects and kernels between those objects. We show how the construction can be used to create kernels from methods that would not normally be viewed as random partitions, such as Random Forest. To demonstrate the potential of this method, we propose two new kernels, the Random Forest Kernel and the Fast Cluster Kernel, and show that these kernels consistently outperform standard kernels on problems involving real-world datasets. Finally, we show how the form of these kernels lend themselves to a natural approximation that is appropriate for certain big data problems, allowing $O(N)$ inference in methods such as Gaussian Processes, Support Vector Machines and Kernel PCA.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Gérard Biau
- Erwan Scornet
- Stefan Wager
- Susan Athey
- Velibor V. Misic
- Guenther Walther
- Gilles Louppe
- Timothy C. Au
In Computer Vision and Pattern Recognition
- Erwan Scornet
- Gérard Biau
- Velibor V. Misic
- Stefan Wager
- Randall Balestriero
- Hristos Tyralis
- Georgia A. Papacharalampous
Citations in the corpus
- A random forest guided tour (2015, background): “, 2009; Denil et al., 2013; Lakshminarayanan et al., 2014). In Lakshminarayanan et al. (2014), so-called Mondrian forests are grown in an online fashion and achieve competitive predictive performance comparable with other online random forests while being faster. When building online forests, a majo”
- Boulevard: Regularized Stochastic Gradient Boosted Trees and Their Limiting Distribution (2018, unknown): “…may improve performance, other simplifications such as the use of completely randomized trees are unlikely to be practically useful, but did allow the development of initial consistency results in Biau (2012) and a connection to kernel methods in Davies and Ghahramani (2014) and Scornet (2016).”