MITRE at SemEval-2018 Task 11: Commonsense Reasoning without Commonsense Knowledge
By Elizabeth M. Merkhofer, John C. Henderson, David Bloom, Laura Strickhart, Guido Zarrella. 2018
16 citations recorded by the source corpus.
This paper describes MITRE’s participation in SemEval-2018 Task 11: Machine Comprehension using Commonsense Knowledge. The techniques explored range from simple bag-of-ngrams classifiers to neural architectures with varied attention and alignment mechanisms. Logistic regression ties the systems together into an ensemble submitted for evaluation. The resulting system answers reading comprehension questions with 82.27% accuracy.
Subfields: Artificial Intelligence, Natural Language Processing, Evaluation & Benchmarks
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Daniel Khashabi
- Dan Roth
- Ashish Sabharwal
- Tushar Khot
- Qiang Ning
- Hannaneh Hajishirzi
- Ben Zhou
- Chitta Baral
In Natural Language Processing
- Daniel Khashabi
- Dan Roth
- Ashish Sabharwal
- Qiang Ning
- Tushar Khot
- Hannaneh Hajishirzi
- Ben Zhou
- Chitta Baral
In Evaluation & Benchmarks
- Dan Roth
- Ben Zhou
- Manfred Pinkal
- Stefan Thater
- Daniel Khashabi
- Simon Ostermann
- Ashutosh Modi
- Shyam Upadhyay
Citations in the corpus
- Looking Beyond the Surface: A Challenge Set for Reading Comprehension over Multiple Sentences (2018, background): “As described in (Merkhofer et al., 2018), this baseline takes into account the lengths of a text, question and each answer candidate, as well as indicator features regarding the (co)occurrences of any words in them.”
- “Going on a vacation” takes longer than “Going for a walk”: A Study of Temporal Commonsense Understanding (2019, background): “This relates our work to a large body of work on question-answering, an area that has seen significant progress in the past few years (Clark et al., 2018; Ostermann et al., 2018; Merkhofer et al., 2018).”
- SemEval-2018 Task 11: Machine Comprehension Using Commonsense Knowledge (2018, methodology): “The top three participating teams, Yuanfudao, MITRE and Jiangnan, all significantly outperform the remaining teams on text-based questions (>80% vs. 74”
- Evidence Extraction for Machine Reading Comprehension with Deep Probabilistic Logic (2018, unknown): “…or Team Name Dev TestTotal Text Common Sense Total Text Common Sense Yuanfudao (Wang, 2018) 85.3 84.0 82.0 85.0 MITRE (Merkhofer et al., 2018) 85.1 82.3 79.0 83.0 ELiRF-UVP (Ostermann et al., 2018) 75.0 82.0 73.0 (Chen et al., 2018) 86.5 84.1 (Ostermann et al., 2018)…”