Understanding Impacts of High-Order Loss Approximations and Features in Deep Learning Interpretation

By Sahil Singla, Eric Wallace, Shi Feng, Soheil Feizi. 2019

63 citations recorded by the source corpus.

Current methods to interpret deep learning models by generating saliency maps generally rely on two key assumptions. First, they use first-order approximations of the loss function neglecting higher-order terms such as the loss curvatures. Second, they evaluate each feature's importance in isolation, ignoring their inter-dependencies. In this work, we study the effect of relaxing these two assumptions. First, by characterizing a closed-form formula for the Hessian matrix of a deep ReLU network, we prove that, for a classification problem with a large number of classes, if an input has a high confidence classification score, the inclusion of the Hessian term has small impacts in the final solution. We prove this result by showing that in this case the Hessian matrix is approximately of rank one and its leading eigenvector is almost parallel to the gradient of the loss function. Our empirical experiments on ImageNet samples are consistent with our theory. This result can have implications in other related problems such as adversarial examples as well. Second, we compute the importance of group-features in deep learning interpretation by introducing a sparsity regularization term. We use the $L_0-L_1$ relaxation technique along with the proximal gradient descent to have an efficient computation of group feature importance scores. Our empirical results indicate that considering group features can improve deep learning interpretation significantly.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Adrian Weller
2. Umang Bhatt
3. José M. F. Moura
4. Dawn Song
5. Lun Wang
6. Xinyu Xing
7. Mengzhou Li
8. Jinjun Xiong

Citations in the corpus

• Explainable machine learning in deployment (2019, background): “s. is theimportanceof(or attributionfor)featurexi ofx. These explanation functions roughly fall into two categories: perturbation-based techniques [8, 14, 22, 34, 50, 61] and gradientbased techniques [7, 41, 57, 59, 60, 62]. Note that gradient-based techniques can be seen as a special case of a pert”
• On Interpretability of Artificial Neural Networks: A Survey (2020, background): “K. Simonyan et al. [157], D. Smilkov et al. [161], M. Sundararajan et al. [168] and S. Singla et al. [160] utilized the idea of gradients to probe the saliency of an input.”
• TABOR: A Highly Accurate Approach to Inspecting and Restoring Trojan Backdoors in AI Systems (2019, methodology): “To address this problem and improve fidelity for resolved triggers, we design a regularization term by borrowing the idea of those explainable AI techniques ( e.g., [2, 10, 14, 19, 20, 35, 37, 39, 44]).”