Towards Interpretable Deep Neural Networks by Leveraging Adversarial Examples

By Yinpeng Dong, Hang Su, Jun Zhu, Fan Bao. 2017

133 citations recorded by the source corpus.

Sometimes it is not enough for a DNN to produce an outcome. For example, in applications such as healthcare, users need to understand the rationale of the decisions. Therefore, it is imperative to develop algorithms to learn models with good interpretability (Doshi-Velez 2017). An important factor that leads to the lack of interpretability of DNNs is the ambiguity of neurons, where a neuron may fire for various unrelated concepts. This work aims to increase the interpretability of DNNs on the whole image space by reducing the ambiguity of neurons. In this paper, we make the following contributions: 1) We propose a metric to evaluate the consistency level of neurons in a network quantitatively. 2) We find that the learned features of neurons are ambiguous by leveraging adversarial examples. 3) We propose to improve the consistency of neurons on adversarial example subset by an adversarial training algorithm with a consistent loss.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jun Zhu
2. Tianyu Pang
3. Hang Su
4. Yinpeng Dong
5. Song-Chun Zhu
6. Cho-Jui Hsieh
7. Ying Nian Wu
8. Huan Zhang

Citations in the corpus

• Unsupervised Learning of Neural Networks to Explain Neural Networks (2018, background): “Many studies for network interpretability mainly visualized image appearance corresponding to a neural unit inside a CNN [39, 22, 29, 7, 38, 6] or extracted image regions that were responsible for network output [24, 8, 14, 9, 27, 17].”
• Explanation Methods in Deep Learning: Users, Values, Concerns and Challenges (2018, background): “More importantly Dong et al. (2017a) show that by training DNNs with adversarial data and a consistent loss, we can trace back errors made by the DNN to individual neurons and identify whether the data was adversarial.”
• ConvNets and ImageNet Beyond Accuracy: Explanations, Bias Detection, Adversarial Examples and Model Criticism (2017, background): “In order to promote the adoption of such algorithms and to foster their positive technological impact, recent studies have been focusing on understanding a model’s behavior from a human point of view [30, 39, 31, 10].”
• EAD: Elastic-Net Attacks to Deep Neural Networks via Adversarial Examples (2017, methodology): “They have also been used in interpreting DNNs (Koh and Liang 2017; Dong et al. 2017).”
• Data-Free Learning of Student Networks (2019, background): “A large number of works have investigated the interpretability of deep neural networks [30, 22, 4].”
• ConvNets and ImageNet Beyond Accuracy: Understanding Mistakes and Uncovering Biases (2017, background): “In order to promote the adoption of such algorithms and to foster their positive technological impact, recent studies have been focusing on understanding a model from the human perspective [40,41,42,43].”
• Comprehensive Review of Artificial Neural Network Applications to Pattern Recognition (2019, background): “[170] found neurons in DNNs did not recognize semantic objects or components but recognize objects or components as periodic base reinforcement.”
• Robust Feature-Level Adversaries are Interpretability Tools (2021, background): “(2017) [11] 4 7 7 7 4 7 4 Geirhos et al.”
• The importance of interpretability and visualization in machine learning for applications in medicine and health care (2019, background): “Some examples include the proposal [10] of an adversarial training scheme where model neurons “are endowed with human-interpretable concepts” and interpretable representations can trace outcomes back to influential neurons, providing an explanation of how models make predictions; the description of ”
• Boosting Adversarial Attacks with Momentum (2017, background): “Among many attempts [13, 3, 15, 10, 24, 17, 11], adversarial training is the most extensively investigated way to increase the robustness of DNNs [5, 10, 24].”
• Interpreting Adversarially Trained Convolutional Neural Networks (2019, methodology): “2017), we have conducted a more systematical investigation for interpreting AT-CNNs.”