A Generalized Knowledge Hunting Framework for the Winograd Schema Challenge
By Ali Emami, Adam Trischler, Kaheer Suleman, Jackie Chi Kit Cheung. 2018
21 citations recorded by the source corpus.
We introduce an automatic system that performs well on two common-sense reasoning tasks, the Winograd Schema Challenge (WSC) and the Choice of Plausible Alternatives (COPA). Problem instances from these tasks require diverse, complex forms of inference and knowledge to solve. Our method uses a knowledge-hunting module to gather text from the web, which serves as evidence for candidate problem resolutions. Given an input problem, our system generates relevant queries to send to a search engine. It extracts and classifies knowledge from the returned results and weighs it to make a resolution. Our approach improves F1 performance on the WSC by 0.16 over the previous best and is competitive with the state-of-the-art on COPA, demonstrating its general applicability.
Subfields: Artificial Intelligence, Natural Language Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yejin Choi
- Ronan Le Bras
- Chandra Bhagavatula
- Hannaneh Hajishirzi
- Mohit Bansal
- Noah A. Smith
- Luke S. Zettlemoyer
- Jackie Chi Kit Cheung
In Natural Language Processing
- Yejin Choi
- Ronan Le Bras
- Chandra Bhagavatula
- Hannaneh Hajishirzi
- Mohit Bansal
- Keisuke Sakaguchi
- Adam Trischler
- Jackie Chi Kit Cheung
Citations in the corpus
- WinoGrande (2019, background): “WSC (Levesque, Davis, and Morgenstern 2011) Liu et al. (2016) 52.8 WKH (Emami et al. 2018) 57.1 Ensemble LMs (Trinh and Le 2018) 63.8 GPT2 (Radford et al. 2019) 70.7 BERT-DPR (cid:63) (Kocijan et al. 2019) 72.2 HNN (He et al. 2019) 75.1 † RoBERTa-DPR (cid:63) (This work) 83.1 RoBERTa-WinoGrande (cid”
- An Adversarial Winograd Schema Challenge at Scale (2019, methodology): “…the WINOGRANDEdebiased on methods/models that have been effective on the original WSC. Wino Knowledge Hunting Wino Knowledge Hunting (WKH) by Emami et al. (2018) is based on an information retrieval approach, where the sentence is parsed into a set of queries and then the model looks for…”
- A Knowledge Hunting Framework for Common Sense Reasoning (2018, methodology): “Extended Work: Previously, Emami et al. (2018) proposed a similar knowledge hunting framework to tackle the Winograd Schema Challenge.”