Cross-Loss Influence Functions to Explain Deep Network Representations
By Andrew Silva, Rohit Chopra, Matthew Craig Gombolay. 2020
20 citations recorded by the source corpus.
As machine learning is increasingly deployed in the real world, it is paramount that we develop the tools necessary to analyze the decision-making of the models we train and deploy to end-users. Recently, researchers have shown that influence functions, a statistical measure of sample impact, can approximate the effects of training samples on classification accuracy for deep neural networks. However, this prior work only applies to supervised learning, where training and testing share an objective function. No approaches currently exist for estimating the influence of unsupervised training examples for deep learning models. To bring explainability to unsupervised and semi-supervised training regimes, we derive the first theoretical and empirical demonstration that influence functions can be extended to handle mismatched training and testing (i.e.,"cross-loss") settings. Our formulation enables us to compute the influence in an unsupervised learning setup, explain cluster memberships, and identify and augment biases in language models. Our experiments show that our cross-loss influence estimates even exceed matched-objective influence estimation relative to ground-truth sample impact.
Subfields: Model Architectures
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Model Architectures
Citations in the corpus
- Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling (2023, background): “…explored in this case study, we think that the finegrained information that Pythia provides on the data seen during training could benefit the promising literature on influence functions to estimate the role of specific training samples on the encoded bias (Brunet et al., 2019; Silva et al., 2022).”
- Towards a Comprehensive Understanding and Accurate Evaluation of Societal Biases in Pre-Trained Transformers (2021, background): “After the seminal work of Bolukbasi et al. (2016), bias has been found ubiquitous in word embedding models (Amorim et al., 2018; Brunet et al., 2018; Rudinger et al., 2018; Zhao et al., 2017; Costa-jussà et al., 2019; Silva et al., 2020).”