Efficient posterior sampling for high-dimensional imbalanced logistic regression.

By Deborshee Sen, Matthias Sachs, Jianfeng Lu, David B. Dunson. 2019

15 citations recorded by the source corpus.

Classification with high-dimensional data is of widespread interest and often involves dealing with imbalanced data. Bayesian classification approaches are hampered by the fact that current Markov chain Monte Carlo algorithms for posterior computation become inefficient as the number [Formula: see text] of predictors or the number [Formula: see text] of subjects to classify gets large, because of the increasing computational time per step and worsening mixing rates. One strategy is to employ a gradient-based sampler to improve mixing while using data subsamples to reduce the per-step computational complexity. However, the usual subsampling breaks down when applied to imbalanced data. Instead, we generalize piecewise-deterministic Markov chain Monte Carlo algorithms to include importance-weighted and mini-batch subsampling. These maintain the correct stationary distribution with arbitrarily small subsamples and substantially outperform current competitors. We provide theoretical support for the proposed approach and demonstrate its performance gains in simulated data examples and an application to cancer data.

Subfields: Artificial Intelligence, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

GP-Tree: A Gaussian Process Classifier for Few-Shot Incremental Learning (2021, methodology): “Several studies suggested alternative, more efficient, posterior sampling techniques when the classes are imbalanced or when using sub-samples of the data (Nemeth & Fearnhead, 2020; Sachs et al., 2020; Sen et al., 2020).”
Posterior Computation with the Gibbs Zig-Zag Sampler (2020, background): “An additional challenge is that the upper bounds typically deteriorate as the dimension of the parameter space increases, although this can be mitigated to a certain extent using non-uniform sub-sampling schemes (Sen et al., 2020).”
Stochastic Gradient Markov Chain Monte Carlo (2019, methodology): “We can also use stratified sampling ideas, which try to ensure each subsample is representative of the full data (Sen et al. 2020), or adapt ideas from stochastic optimisation that uses multi-arm bandits to learn a good sampling distribution (Salehi et al.”