Attention-over-Attention Neural Networks for Reading Comprehension

By Yiming Cui, Z. Chen, Si Wei, Shijin Wang, Ting-Wei Liu, Guoping Hu. 2016

455 citations recorded by the source corpus.

Cloze-style reading comprehension is a representative problem in mining relationship between document and query. In this paper, we present a simple but novel model called attention-over-attention reader for better solving cloze-style reading comprehension task. The proposed model aims to place another attention mechanism over the document-level attention and induces “attended attention” for final answer predictions. One advantage of our model is that it is simpler than related works while giving excellent performance. In addition to the primary model, we also propose an N-best re-ranking strategy to double check the validity of the candidates and further improve the performance. Experimental results show that the proposed methods significantly outperform various state-of-the-art systems by a large margin in public datasets, such as CNN and Children’s Book Test.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Question Answering & Reading Comprehension

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

  1. Mohit Bansal
  2. Jianfeng Gao
  3. Percy Liang
  4. Luke S. Zettlemoyer
  5. Caiming Xiong
  6. Richard Socher
  7. Yoshua Bengio
  8. Hannaneh Hajishirzi

In Computer Vision and Pattern Recognition

  1. Luc Van Gool
  2. Dacheng Tao
  3. Jianfeng Gao
  4. Trevor Darrell
  5. Xiaogang Wang
  6. Cordelia Schmid
  7. Mohit Bansal
  8. Jitendra Malik

In Natural Language Processing

  1. Mohit Bansal
  2. Jianfeng Gao
  3. Hannaneh Hajishirzi
  4. Caiming Xiong
  5. Richard Socher
  6. Xipeng Qiu
  7. Luke S. Zettlemoyer
  8. Furu Wei

In Question Answering & Reading Comprehension

  1. Xipeng Qiu
  2. Yuxing Peng
  3. Minghao Hu
  4. Zhuosheng Zhang
  5. William W. Cohen
  6. Jianfeng Gao
  7. Ting-Wei Liu
  8. Yi Tay

Citations in the corpus