What and Where: Learn to Plug Adapters via NAS for Multidomain Learning

By Hanbin Zhao, Hao Zeng, Xin Qin, Yongjian Fu, Haibo Wang, Bourahla Omar, Xi Li. 2020

21 citations recorded by the source corpus.

As an important and challenging problem, multidomain learning (MDL) typically seeks a set of effective lightweight domain-specific adapter modules plugged into a common domain-agnostic network. Usually, existing ways of adapter plugging and structure design are handcrafted and fixed for all domains before model learning, resulting in learning inflexibility and computational intensiveness. With this motivation, we propose to learn a data-driven adapter plugging strategy with neural architecture search (NAS), which automatically determines where to plug for those adapter modules. Furthermore, we propose an NAS-adapter module for adapter structure design in an NAS-driven learning scheme, which automatically discovers effective adapter module structures for different domains. Experimental results demonstrate the effectiveness of our MDL model against existing approaches under the conditions of comparable performance.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Efficient Continual Learning with Modular Networks and Task-Driven Priors (2020, background): “There is also a body of literature on modular networks for multi-task and multi-domain learning (Ruder et al., 2019; Rebufﬁ et al., 2017; Zhao et al., 2020).”
Memory-Efficient Class-Incremental Learning for Image Classification (2020, unknown): “Architectural strategies [62]–[70] keep the learned knowledge from the old classes and acquire the new knowledge from the new classes by manipulating the network architecture, e.”