Subword-augmented Embedding for Cloze Reading Comprehension

By Zhuosheng Zhang, Yafang Huang, Zhao Hai. 2018

32 citations recorded by the source corpus.

Representation learning is the foundation of machine reading comprehension. In state-of-the-art models, deep learning methods broadly use word and character level representations. However, character is not naturally the minimal linguistic unit. In addition, with a simple concatenation of character and word embedding, previous models actually give suboptimal solution. In this paper, we propose to use subword rather than character for word embedding enhancement. We also empirically explore different augmentation strategies on subword-augmented embedding to enhance the cloze-style reading comprehension model (reader). In detail, we present a reader that uses subword-level representation to augment word embedding with a short list to handle rare words effectively. A thorough examination is conducted to evaluate the comprehensive performance and generalization ability of the proposed reader. Experimental results show that the proposed approach helps the reader significantly outperform the state-of-the-art baselines on various public datasets.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Zhuosheng Zhang
2. Hai Zhao
3. Yuwei Wu
4. Zhao Hai
5. Xi Zhou
6. Xiaoping Zhou
7. Junru Zhou
8. Rui Wang

In Natural Language Processing

1. Zhuosheng Zhang
2. Hai Zhao
3. Yuwei Wu
4. Zhao Hai
5. Xi Zhou
6. Xiaoping Zhou
7. Junru Zhou
8. Shuailiang Zhang

Citations in the corpus

• Dual Co-Matching Network for Multi-choice Reading Comprehension (2019, background): “Machine reading comprehension (MRC) is a fundamental and long-standing goal of natural language understanding which aims to teach machine to answer question automatically according to given passage (Hermann et al. 2015; Ra-jpurkar et al. 2016; Nguyen et al. 2016; Zhang et al. 2018).”
• I Know What You Want: Semantic Learning for Text Comprehension (2018, methodology): “In the last decade, the MRC tasks have evolved from the early cloze-style test (Hill et al. 2015; Hermann et al. 2015; Zhang, Huang, and Zhao 2018) to span-based answer extraction from passage (Rajpurkar et al. 2016; Rajpurkar, Jia, and Liang 2018).”
• Effective Character-Augmented Word Embedding for Machine Reading Comprehension (2018, background): “Machine reading comprehension has been witnessed rapid progress in recent years [26,27,29,22,28,8,33,35,31].”
• Modeling Multi-turn Conversation with Deep Utterance Aggregation (2018, background): “due to their promising potentials and alluring commercial values (Qiu et al., 2017; Cui et al., 2017; Yan et al., 2017; Huang et al., 2018; Jia and Zhao, 2014). With the development of neural models (Zhang et al., 2018c; He et al., 2018; Li et al., 2018; Cai et al., 2018; Zhang and Zhao, 2018), buil”
• SG-Net: Syntax-Guided Machine Reading Comprehension (2019, methodology): “In the last decade, the MRC tasks have evolved from the early cloze-style test (Hill et al., 2015; Hermann et al., 2015; Zhang et al., 2018) to span-based answer extraction from passage (Rajpurkar et al.”