Exploring Shared Structures and Hierarchies for Multiple NLP Tasks
By Junkun Chen, Kaiyu Chen, Xinchi Chen, Xipeng Qiu, Xuanjing Huang. 2018
16 citations recorded by the source corpus.
Designing shared neural architecture plays an important role in multi-task learning. The challenge is that finding an optimal sharing scheme heavily relies on the expert knowledge and is not scalable to a large number of diverse tasks. Inspired by the promising work of neural architecture search (NAS), we apply reinforcement learning to automatically find possible shared architecture for multi-task learning. Specifically, we use a controller to select from a set of shareable modules and assemble a task-specific architecture, and repeat the same procedure for other tasks. The controller is trained with reinforcement learning to maximize the expected accuracies for all tasks. We conduct extensive experiments on two types of tasks, text classification and sequence labeling, which demonstrate the benefits of our approach.
Subfields: Natural Language Processing, Model Architectures
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Natural Language Processing
In Model Architectures
Citations in the corpus
- AdaShare: Learning What To Share For Efficient Deep Multi-Task Learning (2019, methodology): “Some recent works [8, 29], in NLP and character recognition, also try to learn the multi-task sharing via reinforcement learning or evolutionary computation.”
- Multi-Task Learning with Deep Neural Networks: A Survey (2020, methodology): “This method bears a strong resemblance to the previously discussed Multi-task Neural Model Search controller (Wong and Gesmundo, 2017), with the main difference being that the RNN controller used in (Chen et al., 2018) simply chooses between a set of network modules, while the MNMS controller makes ”