Non-Linear Regression for Bag-of-Words Data via Gaussian Process Latent Variable Set Model
By Yuya Yoshikawa, Tomoharu Iwata, Hiroshi Sawada. 2015
12 citations recorded by the source corpus.
Gaussian process (GP) regression is a widely used method for non-linear prediction.The performance of the GP regression depends on whether it can properly capture the covariance structure of target variables, which is represented by kernels between input data.However, when the input is represented as a set of features, e.g. bag-of-words, it is difficult to calculate desirable kernel values because the co-occurrence of different but relevant words cannot be reflected in the kernel calculation.To overcome this problem, we propose a Gaussian process latent variable set model (GP-LVSM), which is a non-linear regression model effective for bag-of-words data.With the GP-LVSM, a latent vector is associated with each word, and each document is represented as a distribution of the latent vectors for words appearing in the document. We efficiently represent the distributions by using the framework of kernel embeddings of distributions that can hold high-order moment information of distributions without need for explicit density estimation.By learning latent vectors so as to maximize the posterior probability, kernels that reflect relations between words are obtained, and also words are visualized in a low-dimensional space.In experiments using 25 item review datasets, we demonstrate the effectiveness of the GP-LVSM in prediction and visualization.
Subfields: Artificial Intelligence, Materials Chemistry, Probabilistic & Bayesian Methods
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Bharath K. Sriperumbudur
- Krikamol Muandet
- Kenji Fukumizu
- Bernhard Schölkopf
- Arthur Gretton
- David Lopez-Paz
- Dino Sejdinovic
- Jan Peters
In Probabilistic & Bayesian Methods
- Krikamol Muandet
- Bharath K. Sriperumbudur
- Kenji Fukumizu
- Bernhard Schölkopf
- Arthur Gretton
- Dino Sejdinovic
- Yu Nishiyama
- Le Song
Citations in the corpus
- Kernel Mean Embedding of Distributions: A Review and Beyonds (2016, background): “This is ubiquitous in, for example, multiple-instance learning (Doran 2013), learning with noisy and uncertain input, learning from missing data, group anomaly detection (Muandet and Schölkopf 2013, Guevara et al. 2015), dataset squishing, and bag-of-words data (Yoshikawa et al. 2014; 2015).”