On‐line expectation–maximization algorithm for latent data models

By Olivier Capp'e, Eric Moulines. arXiv.org, 2007

516 citations recorded by the source corpus.

Summary. We propose a generic on‐line (also sometimes called adaptive or recursive) version of the expectation–maximization (EM) algorithm applicable to latent variable models of independent observations. Compared with the algorithm of Titterington, this approach is more directly connected to the usual EM algorithm and does not rely on integration with respect to the complete‐data distribution. The resulting algorithm is usually simpler and is shown to achieve convergence to the stationary points of the Kullback–Leibler divergence between the marginal distribution of the observation and the model distribution at the optimal rate, i.e. that of the maximum likelihood estimator. In addition, the approach proposed is also suitable for conditional (or regression) models, as illustrated in the case of the mixture of linear regressions model.

Subfields: Computer Networks and Communications, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/3d6a4c1e87fce6abb4f3d34f6cf860cc11e47b23

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Statistics and Probability

Citations in the corpus

RNA Sequencing Data: Hitchhiker's Guide to Expression Analysis (2018, background): “, onlineEM), rather than the factorization of the underlying likelihood function [82] [83] .”
Heterogeneous Federated Learning: State-of-the-art and Research Challenges (2023, methodology): “Then they propose FeSEM, which employs Stochastic Expectation Maximization (SEM) [18] optimization to calculate the distance between the local models and the cluster centers to ensure that an optimal match can be derived.”
Multi-center federated learning: clients clustering for better personalization (2020, methodology): “Therefore, we adapt the Stochastic Expectation Maximization (SEM) [61] optimization framework by adding one step, i.”
Opponent Modeling by Expectation–Maximization and Sequence Prediction in Simplified Poker (2017, unknown): “We use what Liang and Klein refer to as a stepwise EM algorithm [31], first proposed by Sato and Ishii [32], generalised by Cappé and Moulines [33], and applied to poker by Butterworth [34].”
Deep Classifiers from Image Tags in the Wild (2015, methodology): “The method is based on previous robust logistic regression methods [23], and we adapt these methods to the large-scale setting using Stochastic EM [3].”
Stochastic Majorization-Minimization Algorithms for Large-Scale Optimization (2013, background): “For such objectives, online techniques based on stochastic approximations have proven to be particularly efﬁcient, and have drawn a lot of attraction in machine learning, statistics, and optimization [3–6,9–12,14,16,17,19,22,24–26,30].”
Statistical guarantees for the EM algorithm: From population to sample-based analysis (2014, methodology): “2, with more technical aspects deferred to the Supplementary Material ([3], Appendix E).”
Variational Approximations for Generalized Linear Latent Variable Models (2017, unknown): “…numerical integration methods such as adaptive quadrature (Cagnone and Monari, 2013), and the expectation-maximization (EM) algorithm or some variant of it (Sammel et al., 1997; Capp´e and Moulines, 2009); see Skrondal and Rabe-Hesketh (2004) for a thorough review of estimation methods for GLLVMs.”