Construction of an annotated corpus to support biomedical information extraction

By Paul Thompson, Syed A. Iqbal, John McNaught, Sophia Ananiadou. 2009

138 citations recorded by the source corpus.

BackgroundInformation Extraction (IE) is a component of text mining that facilitates knowledge discovery by automatically locating instances of interesting biomedical events from huge document collections. As events are usually centred on verbs and nominalised verbs, understanding the syntactic and semantic behaviour of these words is highly important. Corpora annotated with information concerning this behaviour can constitute a valuable resource in the training of IE components and resources.ResultsWe have defined a new scheme for annotating sentence-bound gene regulation events, centred on both verbs and nominalised verbs. For each event instance, all participants (arguments) in the same sentence are identified and assigned a semantic role from a rich set of 13 roles tailored to biomedical research articles, together with a biological concept type linked to the Gene Regulation Ontology. To our knowledge, our scheme is unique within the biomedical field in terms of the range of event arguments identified. Using the scheme, we have created the Gene Regulation Event Corpus (GREC), consisting of 240 MEDLINE abstracts, in which events relating to gene regulation and expression have been annotated by biologists. A novel method of evaluating various different facets of the annotation task showed that average inter-annotator agreement rates fall within the range of 66% - 90%.ConclusionThe GREC is a unique resource within the biomedical field, in that it annotates not only core relationships between entities, but also a range of other important details about these relationships, e.g., location, temporal, manner and environmental conditions. As such, it is specifically designed to support bio-specific tool and resource development. It has already been used to acquire semantic frames for inclusion within the BioLexicon (a lexical, terminological resource to aid biomedical text mining). Initial experiments have also shown that the corpus may viably be used to train IE components, such as semantic role labellers. The corpus and annotation guidelines are freely available for academic purposes.

Subfields: Artificial Intelligence, Molecular Biology, Evaluation & Benchmarks, Medical & Health AI

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

In Evaluation & Benchmarks

In Medical & Health AI

Robert Stevens

Citations in the corpus

Findings of the 2014 Workshop on Statistical Machine Translation (2014, background): “GREC (Gene Regulation Event Corpus) is a semantically annotated English corpus of abstracts of biomedical papers (Thompson et al., 2009).”
Machine Translation of Medical Texts in the Khresmoi Project (2014, background): “For English, the main source is the AACT collection of texts from Clin-icalTrials.gov. Smaller resources include: Drug-Bank (Knox et al., 2011), GENIA (Kim et al., 2003), FMA (Rosse and Mejino Jr., 2008), GREC (Thompson et al., 2009), and PIL (Bouayad-Agha et al., 2000).”
MAVEN: A Massive General Domain Event Detection Dataset (2020, background): “…datasets are developed for domain-speciﬁc event extraction on special domains, like the bio-medical domain (Pyysalo et al., 2007; Kim et al., 2008; Thompson et al., 2009; Buyko et al., 2010; N´edellec et al., 2013), literature (Sims et al., 2019), Twitter (Ritter et al., 2012; Guo et al., 2013)…”
Deep learning meets ontologies: experiments to anchor the cardiovascular disease ontology in the biomedical literature (2018, methodology): “Hence, we do not follow studies like Thompson et al. [93], which calculate precision and recall, and use F score (i.e. a metric that combines precision and recall) as a way of calculating inter-annotator agreement.”
Adaptation of machine translation for multilingual information retrieval in the medical domain (2014, background): “policy [63], DrugBank – a bioinformatics and cheminformatics resource describing drugs [64], Gene Regulation Event Corpus (GREC) – a semantically annotated English corpus of abstracts of biomedical texts [65], the GENIA corpus of biomedical literature compiled and annotated within the GENIA project ”
Concept annotation in the CRAFT corpus (2012, methodology): “…corpora, as these are obviously most directly related: ABGene [42], BioInfer [43], the CLEF Corpus [44], the FetchProt Corpus [45], the Fourth i2b2/VA Challenge corpus [46], GENETAG [47], GENIA [48,49], GREC [50], the ITI TXM PPI and TE corpora [51], MedPost [52], the PennBioIE Oncology and CYP v1.”
NCBI disease corpus: A resource for disease name recognition and concept normalization (2014, methodology): “We measured agreement by computing the F-measure between each pair of annotators that worked on the same set of documents, as also noted in [26,39].”