Datamodels: Predicting Predictions from Training Data

By Andrew Ilyas, Sung Min Park, Logan Engstrom, Guillaume Leclerc, Aleksander Madry. 2022

215 citations recorded by the source corpus.

We present a conceptual framework, datamodeling, for analyzing the behavior of a model class in terms of the training data. For any fixed"target"example $x$, training set $S$, and learning algorithm, a datamodel is a parameterized function $2^S \to \mathbb{R}$ that for any subset of $S' \subset S$ -- using only information about which examples of $S$ are contained in $S'$ -- predicts the outcome of training a model on $S'$ and evaluating on $x$. Despite the potential complexity of the underlying process being approximated (e.g., end-to-end training and evaluation of deep neural networks), we show that even simple linear datamodels can successfully predict model outputs. We then demonstrate that datamodels give rise to a variety of applications, such as: accurately predicting the effect of dataset counterfactuals; identifying brittle predictions; finding semantically similar examples; quantifying train-test leakage; and embedding data into a well-behaved and feature-rich representation space. Data for this paper (including pre-computed datamodels as well as raw predictions from four million trained deep neural networks) is available at https://github.com/MadryLab/datamodels-data .

Subfields: Artificial Intelligence, Information Systems, Materials Chemistry, Meta & Continual Learning, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Materials Chemistry

In Meta & Continual Learning

In Statistical Learning Theory

Citations in the corpus

Foundation Models and Fair Use (2023, neutral): “…attribution refers to methods that assign attribution scores to training examples to understand the contribution of individual examples (or group of examples) to (test-time) model predictions (Koh & Liang, 2017; Ghorbani & Zou, 2019; Jia et al., 2019; Pezeshkpour et al., 2021; Ilyas et al., 2022).”
Counterfactual Memorization in Neural Language Models (2021, background): “In a follow up work, Ilyas et al. [2022] showed that it is possible to fit a datamodel to predict the outcome of training a model on a specific training subset and evaluating on a specific input.”
TRAK: Attributing Model Behavior at Scale (2023, unknown): “(2022) as an “oracle” of sorts—it makes accurate counterfactual predictions (and as a result has found downstream utility (Ilyas et al., 2022; Shah et al., 2022; Chang & Jia, 2022)), but is (often prohibitively) costly to compute.”
SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generation (2023, methodology): “In addition, there has also been a growing body of research focused on data-level saliency analyses, often referred to as data attribution (Koh & Liang, 2017; Park et al., 2023; Ilyas et al., 2022).”
LESS: Selecting Influential Data for Targeted Instruction Tuning (2024, methodology): “G. Comparison to Data Models (Ilyas et al., 2022) Concurrent work in Engstrom et al. (2024) applies the datamodels framework (Ilyas et al., 2022) to select pre-training data to induce strong performance on target downstream tasks.”
Distilling Model Failures as Directions in Latent Space (2022, unknown): “…dataset has key implications for machine learning models’ behavior (Feldman, 2019; Carlini et al., 2019; Koh & Liang, 2017; Ghorbani & Zou, 2019; Ilyas et al., 2022), especially as the training environments often deviate from deployment conditions (Rabanser et al., 2019; Koh et al., 2020;…”
DataComp: In search of the next generation of multimodal datasets (2023, methodology): “[63] Andrew Ilyas, Sung Min Park, Logan Engstrom, Guillaume Leclerc, and Aleksander Madry.”
Query-efficient model evaluation using cached responses (2026, unknown): “…this enables combining our method with existing techniques – e.g., using DKPS-based methods to predict performance on IRT-selected subsets (Polo et al., 2024) or on “anchor points” (Vivek et al., 2024) – to potentially compound efficiency gains beyond what either approach achieves alone.”