Meta Multi-Task Learning for Sequence Modeling

By Junkun Chen, Xipeng Qiu, Pengfei Liu, Xuanjing Huang. 2018

93 citations recorded by the source corpus.

Semantic composition functions have been playing a pivotal role in neural representation learning of text sequences. In spite of their success, most existing models suffer from the underfitting problem: they use the same shared compositional function on all the positions in the sequence, thereby lacking expressive power due to incapacity to capture the richness of compositionality. Besides, the composition functions of different tasks are independent and learned from scratch. In this paper, we propose a new sharing scheme of composition function across multiple tasks. Specifically, we use a shared meta-network to capture the meta-knowledge of semantic composition and generate the parameters of the task-specific semantic composition models. We conduct extensive experiments on two types of tasks, text classification and sequence tagging, which demonstrate the benefits of our approach. Besides, we show that the shared meta-knowledge learned by our proposed model can be regarded as off-the-shelf knowledge and easily transferred to new tasks.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yu Zheng
2. Junbo Zhang
3. Yong Yu
4. Rishikesh Jha
5. Andrew McCallum
6. Trapit Bansal
7. Zheyi Pan
8. Weifeng Wang

Citations in the corpus

• Urban Traffic Prediction from Spatio-Temporal Data Using Deep Meta Learning (2019, background): “Recently, [5] proposed meta multi-task learning for NLP tasks, which learns task-specific semantic function by a meta network.”
• Learning to Few-Shot Learn Across Diverse Natural Language Classification Tasks (2019, background): “Chen et al. (2018) learn HyperLSTM (Ha et al., 2016) model in a multi-task setting across various sentiment classification domains and show improvements when transferring to a new domain.”