Optimal a priori balance in the design of controlled experiments

By Nathan Kallus. 2013

103 citations recorded by the source corpus.

We develop a unified theory of designs for controlled experiments that balance baseline covariates a priori (before treatment and before randomization) using the framework of minimax variance and a new method called kernel allocation. We show that any notion of a priori balance must go hand in hand with a notion of structure, since with no structure on the dependence of outcomes on baseline covariates complete randomization (no special covariate balance) is always minimax optimal. Restricting the structure of dependence, either parametrically or non‐parametrically, gives rise to certain covariate imbalance metrics and optimal designs. This recovers many popular imbalance metrics and designs previously developed ad hoc, including randomized block designs, pairwise‐matched allocation and rerandomization. We develop a new design method called kernel allocation based on the optimal design when structure is expressed by using kernels, which can be parametric or non‐parametric. Relying on modern optimization methods, kernel allocation, which ensures nearly perfect covariate balance without biasing estimates under model misspecification, offers sizable advantages in precision and power as demonstrated in a range of real and synthetic examples. We provide strong theoretical guarantees on variance, consistency and rates of convergence and develop special algorithms for design and hypothesis testing.

Subfields: Computational Theory and Mathematics, Management Science and Operations Research, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Theory and Mathematics

In Management Science and Operations Research

In Statistics and Probability

Citations in the corpus

Balanced Policy Evaluation and Learning (2017, positive): “This follows a general approach introduced by [22, 24] of finding optimal balancing weights that optimize a given CMSE objective directly rather than via a plug-in approach.”
Generalized Optimal Matching Methods for Causal Inference (2016, background): “Kernels have many applications in statistics, applied and theoretical [31, 32, 33, 34, 35].”
Policy Evaluation with Latent Confounders via Optimal Balance (2019, methodology): “An alternative to approaches based on fixed formulae for computing the importance weights is to compute weights that optimize an imbalance objective function [1, 15, 17, 18].”
Why Propensity Scores Should Not Be Used for Matching (2019, background): “…for the PSM Paradox is because, after PSM achieves its goals of finding a subset of the data that approximates complete randomization, with approximately constant propensity scores within strata, any further pruning is being done essentially at random, exactly as a completely randomized experiment.”