Expectation-Maximization Gaussian-Mixture Approximate Message Passing

By Jeremy P. Vila, Philip Schniter. IEEE Transactions on Signal Processing, 2012

423 citations recorded by the source corpus.

When recovering a sparse signal from noisy compressive linear measurements, the distribution of the signal's non-zero coefficients can have a profound effect on recovery mean-squared error (MSE). If this distribution was a priori known, then one could use computationally efficient approximate message passing (AMP) techniques for nearly minimum MSE (MMSE) recovery. In practice, however, the distribution is unknown, motivating the use of robust algorithms like LASSO-which is nearly minimax optimal-at the cost of significantly larger MSE for non-least-favorable distributions. As an alternative, we propose an empirical-Bayesian technique that simultaneously learns the signal distribution while MMSE-recovering the signal-according to the learned distribution-using AMP. In particular, we model the non-zero distribution as a Gaussian mixture and learn its parameters through expectation maximization, using AMP to implement the expectation step. Numerical experiments on a wide range of signal classes confirm the state-of-the-art performance of our approach, in both reconstruction error and runtime, in the high-dimensional regime, for most (but not all) sensing operators.

Subfields: Computational Mechanics, Computer Networks and Communications, Signal Processing

View Paper: https://www.semanticscholar.org/paper/31bf40a5e69201f8c5f421c6bfda573cc97bc1ae

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Networks and Communications

In Signal Processing

Citations in the corpus

Channel Estimation in Broadband Millimeter Wave MIMO Systems With Few-Bit ADCs (2016, methodology): “The expressions for EX |R [xi | r̂1 ; νr ,θ] and VarX |R [xi | r̂i ; νr ,θ] needed in lines 12-13 of Algorithm 1 and lines 5-6 of Algorithm 2 can be found in [27], as can the EM-update expressions for θ.”
Iterative Channel Estimation Using LSE and Sparse Message Passing for MmWave MIMO Systems (2016, methodology): “The algorithms in the third category do not need any priori knowledge of the channel distribution except the noise variance, e.g., Sparse Bayesian Learning (SBL) [30], LASSO [32], Expectation-maximization Bernoulli-Gaussian Approximate Message Passing (EM-BG-AMP) [33], etc.”
Compressive phase retrieval via generalized approximate message passing (2012, methodology): “Although the noise variance μ may not be known apriori, it can be learned from the observations through an expectation-maximization (EM) [31] procedure, similar to that proposed in [32].”
Optimal errors and phase transitions in high-dimensional generalized linear models (2017, background): “For a modern derivation in the context of linear models, see [11,68,78].”
Compressed Sensing for Energy-Efficient Wireless Telemonitoring of Noninvasive Fetal ECG Via Block Sparse Bayesian Learning (2012, positive): “They were CoSaMP [27], Elastic-Net [28], Basis Pursuit [29], SL0 [30], and EM-GMAMP [31] (with the “heavy-tailed” mode).”
Compressed Channel Estimation for Intelligent Reflecting Surface-Assisted Millimeter Wave Systems (2020, methodology): “2 depicts the NMSEs and ARSPRs versus the SNR, where we set T = 110 for GAMP, OMP and the oracle LS estimator, and T = 1524 > NM for the conventional LS estimator [8].”
Orthogonal AMP (2016, methodology): “To approach the optimal performance for OAMP, the EM learning framework [34] or the parametric SURE approach [37] developed for AMP could be applicable to OAMP as well [38].”
AMP-Inspired Deep Networks for Sparse Linear Inverse Problems (2016, neutral): “ has the BG prior p(x;γ,φ)=(1−γ)δ(x)+γN(x;0,φ) (37) (with γ ∈(0,1) and φ > 0) and r is an AWGN-corrupted measurement of x: r =x +e for e ∼N(0,σ2). (38) The MSE-optimal denoiser is then (see, e.g., [39]) xb= r 1+σ 2 φ 1+1−γ γ N(r;0,σ ) N(r;0,σ2+φ) . (39) To turn (39) into a learnable shrinkage f”
Generalized approximate message passing for estimation with random linear mixing (2010, background): “(51) Since the computations in (53) and (54) are expectations over scalar random variables, they can be evaluated numerically given functions gin(·) and gout(·).”
Compressive Sensing-Based Adaptive Active User Detection and Channel Estimation: Massive Access Meets Massive MIMO (2019, methodology): “The a priori distribution model in (17) originates from the AMP literature [35]–[37], and has been successfully employed for various AMP-based channel estimation schemes [30]–[32].”