Incorporating Priors with Feature Attribution on Text Classification

By Frederick Liu, Besim Avci. 2019

133 citations recorded by the source corpus.

Feature attribution methods, proposed recently, help users interpret the predictions of complex models. Our approach integrates feature attributions into the objective function to allow machine learning practitioners to incorporate priors in model building. To demonstrate the effectiveness our technique, we apply it to two tasks: (1) mitigating unintended bias in text classifiers by neutralizing identity terms; (2) improving classifier performance in scarce data setting by forcing model to focus on toxic terms. Our approach adds an L2 distance loss between feature attributions and task-specific prior values to the objective. Our experiments show that i) a classifier trained with our technique reduces undesired model biases without a tradeoff on the original task; ii) incorporating prior helps model performance in scarce data settings.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Refining Language Models with Compositional Explanations (2021, background): “Despite these initial successes, existing model regularization methods have limited capacity in conveying complex human feedback regarding spurious patterns, and limited regularization strength as the regularization term is enforced on only instances associated with human feedback (while the vast am”
Interpretations are useful: penalizing explanations to align neural networks with prior knowledge (2019, background): “Two recent papers extend these ideas by penalizing gradient-based attributions for natural language models (Liu & Avci, 2019) and to produce smooth attributions (Erion et al., 2019).”
Improving performance of deep learning models with axiomatic attribution priors and expected gradients (2020, methodology): “While feature attributions have previously been used in training (more details in ‘Previous attribution priors’ in Methods) 9,12 , our approach offers two novel components.”
The Unreliability of Explanations in Few-Shot In-Context Learning (2022, background): “Another line of work seeks to aligning the reasoning process of a trained models with the explanations, which is typically done by interpreting a prediction post-hoc through explanation techniques and optimize the distance between the obtained explanation and ground truth explanation (Liu and Avci, ”
Human-in-the-loop Debugging Deep Text Classifiers (2020, background): “With a limited number of training examples, using human rationales or prior knowledge together with training labels can help the models perform better (Zaidan et al., 2007; Bao et al., 2018; Liu and Avci, 2019).”
The Unreliability of Explanations in Few-shot Prompting for Textual Reasoning (2022, background): “Another line of work seeks to align the reasoning process of a trained model with the explanations, which is typically done by interpreting a prediction post-hoc through explanation techniques and optimizing the distance between the obtained explanation and ground truth explanation (Liu and Avci, 20”
Explaining Patterns in Data with Language Models via Interpretable Autoprompting (2022, background): “Alternatively, one can investigate whether an LLM’s learned representations via probing [37, 38] or by directly analyzing a model’s internal weights and activations [39–41].”
Why Attention is Not Explanation: Surgical Intervention and Causal Reasoning about Neural Models (2020, background): “…generated plaintext, partially or entirely independent of the actual classiﬁcation or factual content but facially plausible (Xu et al., 2015b; Liu et al., 2019); or direct exposure of structure in the underlying model, such as traversals through a graph (Yang et al., 2018; Moon et al., 2019).”
The Birth of Bias: A case study on the evolution of gender bias in an English language model (2022, unknown): “), the training procedure (Zhang et al., 2018; Zhao et al., 2018b; Liu and Avci, 2019), or on measuring and fixing biases in word embeddings or internal states of language models (Bolukbasi et al.”
Fair Hate Speech Detection through Evaluation of Social Group Counterfactuals (2020, methodology): “Other approaches modify training objectives via L2 norms of feature importance (Liu and Avci 2019) or via regularization of post-hoc term-level importance (Kennedy et al. 2020b).”