A framework for benchmarking entity-annotation systems

By Marco Cornolti, Paolo Ferragina, Massimiliano Ciaramita. 2013

227 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Information Systems, Sociology and Political Science, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Sociology and Political Science

In Evaluation & Benchmarks

Citations in the corpus

Learning Dense Representations for Entity Retrieval (2019, background): “These variations and the resulting fragmentation of evaluation is discussed at length by Ling et al. (2015) and Hachey et al. (2013), and partially addressed by attempts to consolidate datasets (Cornolti et al., 2013) and metrics (Usbeck et al., 2015).”
Entity Linking with a Knowledge Base: Issues, Techniques, and Solutions (2015, background): “[137] recently proposed a publicly available”
Document Retrieval Model Through Semantic Linking (2017, methodology): “Cornolti et al. [6] provide a benchmarking framework for comparing annotation systems and analyse some of the popular systems such as TagMe [9], Wikipedia-miner [23], and DBpedia Spotlight [20].”
Entity Linking meets Word Sense Disambiguation: a Unified Approach (2014, methodology): “For the KORE50 and AIDA-CoNLL datasets we compared our sys-tem with six approaches, including state-of-the-art ones (Hoffart et al., 2012; Cornolti et al., 2013): • MW, i.e., the Normalized Google Distance as deﬁned by Milne and Witten (2008); • KPCS (Hoffart et al., 2012), which calculates a…”
Conducting systematic literature reviews and bibliometric analyses (2019, background): “…used catalogues for entity linking is Wikipedia, as it covers a vast and ever-increasing number of entities, has wide content coverage and includes features such as disambiguation pages and unique identifiers for each page (Cornolti et al., 2013; Ferragina and Scaiella, 2010; Khalid et al., 2008).”
GERBIL - Benchmarking Named Entity Recognition and Linking consistently (2017, methodology): “G ERBIL comes with six measures subdivided into two groups derived from the BAT-framework, namely the micro-and the macro-group of precision, recall and f-measure.”
From TagME to WAT: a new entity annotator (2014, methodology): “In this paper we propose a novel entity annotator for texts which hinges on Tagme’s algorithmic technology, currently the best one available [6, 4].”
Document Retrieval Using Entity-Based Language Models (2016, methodology): “In Section 4.2.1 we also show the eﬀective-ness of our methods using the Wikiﬁer entity-linking tool [9, 12].”
Design Challenges for Entity Linking (2015, background): “We note that Bag-of-Concept F1 is equivalent to the measure for Concept-to-Wikipedia task proposed in (Cornolti et al., 2013) and NER-style F1 is the same as strong annotation match.”