Are Neural Nets Modular? Inspecting Functional Modularity Through Differentiable Weight Masks

By R'obert Csord'as, Sjoerd van Steenkiste, J. Urgen Schmidhuber. 2020

120 citations recorded by the source corpus.

Neural networks (NNs) whose subnetworks implement reusable functions are expected to offer numerous advantages, including compositionality through efficient recombination of functional building blocks, interpretability, preventing catastrophic interference, etc. Understanding if and how NNs are modular could provide insights into how to improve them. Current inspection methods, however, fail to link modules to their functionality. In this paper, we present a novel method based on learning binary weight masks to identify individual weights and subnets responsible for specific functions. Using this powerful tool, we contribute an extensive study of emerging modularity in NNs that covers several standard architectures and datasets. We demonstrate how common NNs fail to reuse submodules and offer new insights into the related issue of systematic generalization on language tasks.

Subfields: Artificial Intelligence, Model Architectures, Safety, Robustness & Interpretability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Model Architectures

In Safety, Robustness & Interpretability

Citations in the corpus

Editing Factual Knowledge in Language Models (2021, background): “Causal Interventions Identiﬁcation of minimal changes to neural networks needed to achieve a certain behaviour has been studied in the context of research in interpreting neural networks (Lakretz et al., 2019; Vig et al., 2020; Elazar et al., 2021; Csordás et al., 2021).”
Continual Learning via Local Module Composition (2021, background): “Modularity in neural networks is studied in the context of scalability [9], and more recently as a way to achieve compositionality and systematic generalization [6, 47, 15, 8, 27, 19] as well as for multi-task learning [65, 82].”
Memory-Based Model Editing at Scale (2022, background): “with potentially disjoint parts of the model being responsible for predictions on different inputs (Csordás et al., 2021).”
How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition (2022, background): “Given the apparent importance of modularity and compositionality, Csordás et al. (2021) studied two properties of neural networks without modular architectures: whether they automatically learn specialized modules, and whether they reuse those modules.”
Emergent Structures and Training Dynamics in Large Language Models (2022, background): “With these findings, Csordás et al. (2021) suggest further research about “function dependent weight shar-579 ing in the neural networks”.”