Maximin effects in inhomogeneous large-scale data

By Nicolai Meinshausen, Peter Buhlmann. 2014

101 citations recorded by the source corpus.

Large-scale data are often characterised by some degree of inhomogeneity as data are either recorded in dierent time regimes or taken from multiple sources. We look at regression models and the eect of randomly changing coecients, where the change is either smoothly in time or some other dimension or even without any such structure. Fitting varying-coecient models or mixture models can be appropriate solutions but are computationally very demanding and often try to return more information than necessary. If we just ask for a model estimator that shows good predictive properties for all regimes of the data, then we are aiming for a simple linear model that is reliable for all possible subsets of the data. We propose a maximin eects estimator and look at its prediction accuracy from a theoretical point of view in a mixture model with known or unknown group structure. Under certain circumstances the estimator can be computed orders of magnitudes faster than standard penalised regression estimators, making computations on large-scale data feasible. Empirical examples complement the novel methodology and theory.

Subfields: Artificial Intelligence, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Invariant Risk Minimization (2019, background): “For example, [37] selects re = V[Y ] to maximize the minimal explained variance across environments.”
Anchor regression: Heterogeneous data meet causality (2018, methodology): “Approaches to deal with inhomogeneities include robust methods [Huber, 1964, 1973], mixed effects models [Pinheiro and Bates, 2000], time-varying coefficient models [Hastie and Tibshirani, 1993, Fan and Zhang, 1999] and maximin effects [Meinshausen and Bühlmann, 2015].”
Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization (2019, methodology): “For example, the maximin regression framework (Meinshausen & B ¨ uhlmann, 2015) also assumes group-based shifts, but focuses on settings without the generalization problems identiﬁed in our work.”
Out-of-Distribution Generalization via Risk Extrapolation (REx) (2020, background): “The use of “baselines” designed to account for these differences exists in the literature (Meinshausen et al., 2015), but the baseline proposed by Meinshausen et al. (2015), V ar ( Y ) , only provides an upper bound on the true noise-level.”
Learning Models with Uniform Performance via Distributionally Robust Optimization (2018, background): “as a benchmark, where Σ̂n,g is the empirical covariance matrix of the Xi with Gi = g, which maximizes the explained variance for each group [67].”
Robustness to Spurious Correlations via Human Annotations (2020, methodology): “Although our work draws on ideas from domain adaptation (Mansour et al., 2009a;b; Ben-David et al., 2006) causal invariance (Peters et al., 2016; Meinshausen & B¨uhlmann, 2015; Rothenh¨ausler et al., 2018), and robust optimization (Ben-Tal et al., 2013; Duchi & Namkoong, 2018; Bertsimas et al.,…”