Learning Credible Deep Neural Networks with Rationale Regularization

By Mengnan Du, Ninghao Liu, Fan Yang, Xia Hu. 2019

47 citations recorded by the source corpus.

Recent explainability related studies have shown that state-of-the-art DNNs do not always adopt correct evidences to make decisions. It not only hampers their generalization but also makes them less likely to be trusted by end-users. In pursuit of developing more credible DNNs, in this paper we propose CREX, which encourages DNN models to focus more on evidences that actually matter for the task at hand, and to avoid overfitting to data-dependent bias and artifacts. Specifically, CREX regularizes the training process of DNNs with rationales, i.e., a subset of features highlighted by domain experts as justifications for predictions, to enforce DNNs to generate local explanations that conform with expert rationales. Even when rationales are not available, CREX still could be useful by requiring the generated explanations to be sparse. Experimental results on two text classification datasets demonstrate the increased credibility of DNNs trained with CREX. Comprehensive analysis further shows that while CREX does not always improve prediction accuracy on the held-out test set, it significantly increases DNN accuracy on new and previously unseen data beyond test set, highlighting the advantage of the increased credibility.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Interpretations are useful: penalizing explanations to align neural networks with prior knowledge (2019, background): “An alternative line of work proposes penalizing the gradients of a neural network to match human-provided binary annotations and shows the possibility to improve performance (Ross et al., 2017; Bao et al., 2018; Du et al., 2019) and adversarial robustness (Ross & Doshi-Velez, 2018).”
A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems (2018, methodology): “tion map with a higher level of granularity compared to object segmentation maps. Similarly, feature-level annotations have been used as the explanation ground truth in the text classification domain [53]. Other less accurate means of feature attribution like bounding box in images datasets have bee”