A review of modularization techniques in artificial neural networks

By Mohammed Amer, Tomás Henrique Maul. 2019

93 citations recorded by the source corpus.

Artificial neural networks (ANNs) have achieved significant success in tackling classical and modern machine learning problems. As learning problems grow in scale and complexity, and expand into multi-disciplinary territory, a more modular approach for scaling ANNs will be needed. Modular neural networks (MNNs) are neural networks that embody the concepts and principles of modularity. MNNs adopt a large number of different techniques for achieving modularization. Previous surveys of modularization techniques are relatively scarce in their systematic analysis of MNNs, focusing mostly on empirical comparisons and lacking an extensive taxonomical framework. In this review, we aim to establish a solid taxonomy that captures the essential properties and relationships of the different variants of MNNs. Based on an investigation of the different levels at which modularization techniques act, we attempt to provide a universal and systematic framework for theorists studying MNNs, also trying along the way to emphasise the strengths and weaknesses of different modularization approaches in order to highlight good practices for neural network practitioners.

Subfields: Artificial Intelligence, Management Science and Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

Citations in the corpus

Continual Learning via Local Module Composition (2021, background): “In principle, a modular learner capable of composing modules in meaningful structures can provide additional benefits including (i) computational gains due to only executing modules that are relevant to a task [47, 4]; (ii) memory gains due to instantiating a sub-linear number of modules w.”
A survey of the recent architectures of deep convolutional neural networks (2019, background): “e times. This shifted the trend from custom layer design towards modular and uniform layer design. The concept of modularity in CNNs made it easy to tailor them for different tasks effortlessly [31], [100]. In this connection, a different idea of branching and block within a layer was introduced by ”