The GREC Named Entity Generation Challenge 2009: Overview and Evaluation Results
By Anya Belz, Eric Kow, Jette Viethen. 2009
10 citations recorded by the source corpus.
The GREC-NEG Task at Generation Challenges 2009 required participating systems to select coreference chains for all people entities mentioned in short encyclopaedic texts about people collected from Wikipedia. Three teams submitted six systems in total, and we additionally created four baseline systems. Systems were tested automatically using a range of existing intrinsic metrics. We also evaluated systems extrinsically by applying coreference resolution tools to the outputs and measuring the success of the tools. In addition, systems were tested in an intrinsic evaluation involving human judges. This report describes the GREC-NEG Task and the evaluation methods applied, gives brief descriptions of the participating systems, and presents the evaluation results.
Subfields: Artificial Intelligence, Communication, Natural Language Processing, Information Extraction
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Elizabeth Clark
- Jianfeng Gao
- Asli Celikyilmaz
- Laura Perez-Beltrachini
- Claire Gardent
- Anastasia Shimorina
- Shashi Narayan
- Mirella Lapata
In Communication
- Elizabeth Clark
- Asli Celikyilmaz
- Jianfeng Gao
- Eric Kow
- Anya Belz
- Laura Perez-Beltrachini
- Claire Gardent
- Anastasia Shimorina
In Natural Language Processing
- Elizabeth Clark
- Claire Gardent
- Jianfeng Gao
- Laura Perez-Beltrachini
- Asli Celikyilmaz
- Anastasia Shimorina
- Shashi Narayan
- Ondrej Dusek
In Information Extraction
- Laura Perez-Beltrachini
- Elizabeth Clark
- Asli Celikyilmaz
- Jianfeng Gao
- Eric Kow
- Anya Belz
- Claire Gardent
Citations in the corpus
- The WebNLG Challenge: Generating Text from RDF Data (2017, background): “…(NLG) challenges have focused on surface realisation (Banik et al., 2013; Belz et al., 2011), referring expression generation (Belz and Gatt, 2007; Gatt et al., 2008; Gatt et al., 2009; Belz et al., 2008; Belz et al., 2009; Belz et al., 2010) and content selection (BouayadAgha et al., 2013).”
- Evaluation of Text Generation: A Survey (2020, methodology): “The GREC-NEG Task at Generation Challenges 2009 (Belz et al., 2009) evaluated models in select correct coreference chains for all people entities mentioned in short encyclopaedic texts about people collected from Wikipedia.”
- Comparing Rating Scales and Preference Judgements in Language Evaluation (2010, methodology): “This data was used in theGREC-NEG’09 shared-task competition (Belz et al., 2009), where the task was to create systems which automatically select suitableREs for all references to all person entities in a text.”