Bayesian Inference and Optimal Design in the Sparse Linear Model

By Matthias W. Seeger, Florian Steinke, Koji Tsuda. International Conference on Artificial Intelligence and Statistics, 2007

347 citations recorded by the source corpus.

View Paper: https://www.semanticscholar.org/paper/bc43a1d2e609a9878117fb0ea75140271a5d4677

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Expectation Propagation in Gaussian Process Dynamical Systems (2012, methodology): “Expectation Propagation [10, 11] is a widely-used deterministic algorithm for approximate Bayesian inference that has been shown to be highly accurate in many problems, including sparse regression models [17], GP classification [9], and inference in dynamical systems [13, 6, 18].”
Model Selection: Beyond the Bayesian/Frequentist Divide (2010, background): “Optimizing the variational parameters yields the best approximation of the posterior (see e.g., Seeger, 2008).”
Sparse Modeling for Image and Vision Processing (2014, methodology): “A first step towards a Bayesian dictionary learning formulation consists of integrating instead of maximizing with respect to the latent variables αi, as typically done in Bayesian sparse linear models [Park and Casella, 2008, Seeger, 2008], and maximizing the posterior p(D|X).”
Structured sparsity through convex optimization (2011, methodology): “Alternative sparse methods Bayesian methods • Lasso: minimize ∑n i=1 (yi − wxi)(2) + λ‖w‖1 – Equivalent to MAP estimation with Gaussian likelihood and factorized Laplace prior p(w) ∝ ∏p j=1 e −λ|wj| (Seeger, 2008) – However, posterior puts zero weight on exact zeros • Heavy-tailed distributions as a”
A Unifying Framework for Gaussian Process Pseudo-Point Approximations using Power Expectation Propagation (2016, background): “Furthermore, it can be shown to be numerically stable when the factors are log-concave (as in GP regression and classification without pseudo-data) (Seeger, 2008).”
Generalized approximate message passing for estimation with random linear mixing (2010, methodology): “ve appeared under a variety of names including “approximate BP”, “relaxed BP” and, most recently, “approximate message passing”. Gaussian approximations are also used in expectation propagation [32], [33], as well as the analysis and design of LDPC codes [34]–[36]. The GAMP algorithm presented here ”
Sparse Representation for Computer Vision and Pattern Recognition (2010, methodology): “This joint optimization outperforms both the use of random sensing matrices and those matrices that are optimized independently of the learning of the dictionary, Figure 6. Particular cases of the proposed framework include the optimization of the sensing matrix for a given dictionary as well as the”
Expectation propagation for neural networks with sparsity-promoting priors (2013, methodology): “Our aim is to introduce the benefits of the sparse linear models (Seeger, 2008; HernándezLobato et al., 2008) into nonlinear regression by combining the sparse priors with a two-layer NN in a computationally efficient EP framework.”
Predictive Entropy Search for Efficient Global Optimization of Black-box Functions (2014, background): “The variance of the right hand size of (8) is given by where ) , and φ ( · ) and Φ( · ) are the standard Gaussian density function and cdf, respectively.”
Actively Learning what makes a Discrete Sequence Valid (2017, background): “This tendency of active learning methods to perform poorly when they are applied too early during training has already been reported before (Seeger, 2008), and in in our it is likely due to the uncertainty estimates not being well calibrated when the model is randomly initialised.”
Gaussian Kullback-Leibler approximate inference (2013, methodology): “Power G-EP, as proposed in Minka (2004), has previously been shown to have robust convergence for under determined linear models with Laplace potentials (Seeger, 2008).”
Expectation Propagation as a Way of Life: A Framework for Bayesian Inference on Partitioned Data (2014, background): “Power EP has been shown to improve the robustness of the algorithm when the approximation family is not flexible enough (Minka, 2005) or when the propagation of information is difficult due to vague prior information (Seeger, 2008).”
A Unifying Framework for Sparse Gaussian Process Approximation using Power Expectation Propagation (2016, background): “verge but we did not observe any convergence issues in our experiments. Furthermore, it can be shown to be numerically stable when the factors are log-concave (as in GP regression and classication) (Seeger, 2008). When = 1, Power EP is called EP and as !0 the Power EP solution is the minimum of a va”
Efficient Spectral Feature Selection with Minimum Redundancy (2010, methodology): “…i th row of W , and (cid:107) W 2 , 1 is the L 2 , 1 - norm which is deﬁned in the following way: When applied in regression, the L 2 , 1 -norm constraint is equivalent to applying Laplace prior (Seeger 2008) on w i , which tends to force many rows in W to be 0 (cid:62) , resulting sparse solution.”
Learning with Submodular Functions: A Convex Optimization Perspective (2011, background): “Indeed, the marginal likelihood is obtained by the best log-likelihood when maximizing with respect to wA plus the entropy of the covariance matrix [191].”
Lp-Nested Symmetric Distributions (2010, methodology): “Various methods have been proposed to learn W , ranging from sampling the posterior only at its maximum [Olshausen and Field, 1996], approximating the posterior with a Gaussian via the Laplace approximation [Lewicki and Olshausen, 1999] or using Expectation Propagation [Seeger, 2008].”
Graphical Models, Exponential Families, and Variational Inference (2008, background): “[209] reported that reweighted forms of EP appear to have better empirical convergence properties than standard EP.”
Fast nonnegative deconvolution for spike train inference from population calcium imaging. (2009, background): “Note that the prior P [n] acts as a regularizing term, potentially imposing sparseness or smoothness, depending on the assumed distribution [24, 25].”