Evaluating the Inferential Utility of Lexical-Semantic Resources

By Shachar Mirkin, Ido Dagan, Eyal Shnarch. 2009

29 citations recorded by the source corpus.

Lexical-semantic resources are used extensively for applied semantic inference, yet a clear quantitative picture of their current utility and limitations is largely missing. We propose system- and application-independent evaluation and analysis methodologies for resources' performance, and systematically apply them to seven prominent resources. Our findings identify the currently limited recall of available resources, and indicate the potential to improve performance by examining non-standard relation types and by distilling the output of distributional methods. Further, our results stress the need to include auxiliary information regarding the lexical and logical contexts in which a lexical inference is valid, as well as its prior validity likelihood.

Subfields: Artificial Intelligence, Information Retrieval & Search

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Retrieval & Search

Citations in the corpus

From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions (2014, background): “…to lexical similarity (Section 2.1) are very effective at identifying which words are related to the same topic, and can provide useful features for systems that perform semantic inferences (Mirkin et al., 2009), but are not suited to capture precise entailments between complex expressions.”
Knowledge-Based Textual Inference via Parse-Tree Transformations (2015, background): “For a recent empirical study on the inferential utility of common lexical resources, see the work of Mirkin, Dagan, and Shnarch (2009).”
A Survey of Paraphrasing and Textual Entailment Methods (2009, methodology): “Mirkin et al. (2009a) discuss how to evaluate collections of textual entailment rules.”
Types of Common-Sense Knowledge Needed for Recognizing Textual Entailment (2011, background): “These phenomena are well-known to be important to language understanding and RTE (Mirkin et al., 2009; Roth and Sammons, 2007).”
Directional distributional similarity for lexical inference (2010, background): “The first one corresponds to valid entailment relations, both substitutable (Geffet and Dagan 2005) and nonsubstitutable (Mirkin et al. 2009).”
Design and realization of a modular architecture for textual entailment (2013, background): “Various studies have investigated the role of different types of knowledge in textual entailment, such as morphological, syntactic, semantic, and world knowledge (Clark et al. 2007; Mirkin et al. 2009).”