Stochastic Answer Networks for Natural Language Inference

By Xiaodong Liu, Kevin Duh, Jianfeng Gao. 2018

46 citations recorded by the source corpus.

We propose a stochastic answer network (SAN) to explore multi-step inference strategies in Natural Language Inference. Rather than directly predicting the results given the inputs, the model maintains a state and iteratively refines its predictions. Our experiments show that SAN achieves the state-of-the-art results on three benchmarks: Stanford Natural Language Inference (SNLI) dataset, MultiGenre Natural Language Inference (MultiNLI) dataset and Quora Question Pairs dataset.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Jianfeng Gao
2. Weizhu Chen
3. Xiaodong Liu
4. Pengcheng He
5. Mohit Bansal
6. Percy Liang
7. Luke S. Zettlemoyer
8. Yoshua Bengio

In Computer Vision and Pattern Recognition

1. Jianfeng Gao
2. Weizhu Chen
3. Xiaodong Liu
4. Pengcheng He
5. Luc Van Gool
6. Dacheng Tao
7. Xiaogang Wang
8. Trevor Darrell

In Natural Language Processing

1. Jianfeng Gao
2. Xiaodong Liu
3. Weizhu Chen
4. Pengcheng He
5. Jugal Kumar Kalita
6. Kevin Duh
7. Julian R. Medina
8. Dan Otter

Citations in the corpus

• Multi-Task Deep Neural Networks for Natural Language Understanding (2019, methodology): “Two such examples use the SAN answer module for the pairwise text classification output module, and the pairwise ranking loss for the QNLI task which by design is a binary classification 5 There is an ongoing discussion on revising the QNLI dataset.”
• Stochastic Answer Networks for Machine Reading Comprehension (2017, background): “The key difference between the two is what strategies are applied to search the final answers in the neural space.”
• SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization (2019, methodology): “Note that we did not use the complicated answer module, e.g., SAN (Liu et al., 2018). on SST) showing that SMART improves the generalization of MTL.”
• The Microsoft Toolkit of Multi-Task Deep Neural Networks for Natural Language Understanding (2020, methodology): “A multi-step reasoning decoder, SAN (Liu et al., 2018a,b) is also provided.”
• A Survey of the Usages of Deep Learning for Natural Language Processing (2020, methodology): “The model used a stochastic answer network (SAN) [73], [74] ontop of a BERT-like model.”
• Improving Language Understanding by Generative Pre-Training (2018, background): “Although there has been a lot of recent interest [58, 35, 44], the task remains challenging due to the presence of a wide variety of phenomena like lexical entailment, coreference, and lexical and syntactic ambiguity.”