Crowdsourcing Inference-Rule Evaluation

By Naomi Zeichner, Jonathan Berant, Ido Dagan. 2012

44 citations recorded by the source corpus.

Subfields: Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

1. Yejin Choi
2. Noah A. Smith
3. Samuel R. Bowman
4. Mohit Bansal
5. Swabha Swayamdipta
6. Yoav Goldberg
7. Luke S. Zettlemoyer
8. Omer Levy

In Evaluation & Benchmarks

1. Yejin Choi
2. Noah A. Smith
3. Samuel R. Bowman
4. Swabha Swayamdipta
5. Mohit Bansal
6. Omer Levy
7. Roy Schwartz
8. Douwe Kiela

Citations in the corpus

• Annotation Artifacts in Natural Language Inference Data (2018, methodology): “Levy and Dagan (2016) showed that a relation inference benchmark (Zeichner et al., 2012) is severely biased towards distributional methods, since it was created using DIRT (Lin and Pantel, 2001).”
• Annotating Relation Inference in Context via Question Answering (2016, methodology): “This suggests that a massive amount of inference examples, which are necessary for answering questions, are inherently ignored in (Zeichner et al., 2012) and post-hoc evaluations.”