Learning explanations that are hard to vary

By Giambattista Parascandolo, Alexander Neitz, Antonio Orvieto, Luigi Gresele, Bernhard Schölkopf. 2020

229 citations recorded by the source corpus.

In this paper, we investigate the principle that `good explanations are hard to vary' in the context of deep learning. We show that averaging gradients across examples -- akin to a logical OR of patterns -- can favor memorization and `patchwork' solutions that sew together different strategies, instead of identifying invariances. To inspect this, we first formalize a notion of consistency for minima of the loss surface, which measures to what extent a minimum appears only when examples are pooled. We then propose and experimentally validate a simple alternative algorithm based on a logical AND, that focuses on invariances and prevents memorization in a set of real-world tasks. Finally, using a synthetic dataset with a clear distinction between invariant and spurious mechanisms, we dissect learning signals and compare this approach to well-established regularizers.

Subfields: Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Safety, Robustness & Interpretability

Citations in the corpus

Toward Causal Representation Learning (2021, background): “Machine learning models that incorporate or learn structural knowledge of an environment have been shown to be more efficient and generalize better [9], [11], [15], [16], [27], [58], [77], [84], [85], [141], [157], [177], [181], [197], [211], [212], [244], [258], [272], [274].”
Gradient Starvation: A Learning Proclivity in Neural Networks (2020, methodology): “Here, we would like to elaborate on an interesting thought experiment of [79]: Suppose a neural network is provided with a chess book containing examples of chess games with the best movements indicated by a red arrow.”
What shapes feature representations? Exploring datasets, architectures, and training (2020, background): “propose a new approach to optimization that could potentially ameliorate this affect [31].”
Generalizing to Unseen Domains: A Survey on Domain Generalization (2021, background): “(DDC [188], DANN [91], CORAL [191]), and Learning strategy (MLDG [17], RSC [150], GroupDRO [155], ANDMask [256]).”
Gradient Matching for Domain Generalization (2021, background): “In Parascandolo et al. (2021), authors show that by masking out the gradients that have opposite signs in different domains, they are able to trade-off some “learning speed” for prioritizing learning the invariances.”
Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development (2021, methodology): “We then include various types of SOTA domain generalization algorithms: MMD (MaximumMean Discrepancy) (Li et al. 2018) optimizes the similarities of maximum mean discrepancy across domains, CORAL (Correlation Alignment) (Sun & Saenko 2016) matches the mean and covariance of features across domains; ”
Domain Generalization via Gradient Surgery (2021, methodology): “A similar approach was derived in [21] following the notion of invariances , to improve consistency across different domains as the opposite of averaging gradients 1 .”