EGASP: the human ENCODE Genome Annotation Assessment Project

By Roderic Guigó, Paul Flicek, J. F. Abril, Alexandre Reymond, Julien Lagarde, F. Denoeud, Stylianos E. Antonarakis, Michael Ashburner, Vladimir B. Bajic, Ewan Birney and 8 more. Genome Biology, 2006

351 citations recorded by the source corpus.

BackgroundWe present the results of EGASP, a community experiment to assess the state-of-the-art in genome annotation within the ENCODE regions, which span 1% of the human genome sequence. The experiment had two major goals: the assessment of the accuracy of computational methods to predict protein coding genes; and the overall assessment of the completeness of the current human genome annotations as represented in the ENCODE regions. For the computational prediction assessment, eighteen groups contributed gene predictions. We evaluated these submissions against each other based on a 'reference set' of annotations generated as part of the GENCODE project. These annotations were not available to the prediction groups prior to the submission deadline, so that their predictions were blind and an external advisory committee could perform a fair assessment.ResultsThe best methods had at least one gene transcript correctly predicted for close to 70% of the annotated genes. Nevertheless, the multiple transcript accuracy, taking into account alternative splicing, reached only approximately 40% to 50% accuracy. At the coding nucleotide level, the best programs reached an accuracy of 90% in both sensitivity and specificity. Programs relying on mRNA and protein sequences were the most accurate in reproducing the manually curated annotations. Experimental validation shows that only a very small percentage (3.2%) of the selected 221 computationally predicted exons outside of the existing annotation could be verified.ConclusionThis is the first such experiment in human DNA, and we have followed the standards established in a similar experiment, GASP1, in Drosophila melanogaster. We believe the results presented here contribute to the value of ongoing large-scale annotation projects and should guide further experimental methods when being scaled up to the entire human genome sequence.

Subfields: Genetics, Molecular Biology

View Paper: https://www.semanticscholar.org/paper/e5c1388234cff49e90a5f96f54804061464dee84

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Genetics

In Molecular Biology

Citations in the corpus

Apollo: a community resource for genome annotation editing (2009, background): “…support for the generic feature format version 3 (GFF3), continuous transcriptome data, a full Chado database interface, integration with remote services for on-thefly BLAST and Primer BLAST analyses, graphical interfaces for configuring user preferences and full undo of all edit operations.”
The Ensembl gene annotation system (2016, background): “Since the project was launched 16 years ago (1), our gene sets have maintained a reputation as being of the highest quality (2, 3).”
MAKER-P: A Tool Kit for the Rapid Creation, Management, and Quality Control of Plant Genome Annotations1[W][OPEN] (2013, background): “These results demonstrate an established fact of gene ﬁ nding: given suf ﬁ cient training data, good gene-level accuracies are relatively easy to obtain (Guigó et al., 2006; Yandell and Ence, 2012).”
An international effort towards developing standards for best practices in analysis, interpretation and reporting of clinical genome sequencing results in the CLARITY Challenge (2014, methodology): “Successful competitions have accelerated progress in protein folding, including the MATLAB Protein Folding Contest [15] and the International Protein Folding Competition (CASP) [16], gene identification, such as EGASP [17], and in silico tools for predicting variant pathogenicity such as the CAGI ex”
Identification and analysis of functional elements in 1% of the human genome by the ENCODE pilot project (2007, background): “3), and integrated annotation of available complementary DNA and EST sequences involving computational, manual, and experimental approaches 16 (see Supplementary Information section 2.4).”
Ensembl 2007 (2006, methodology): “The EGASP gene prediction competition (22) was held to compare a variety of automatic gene prediction methods with a curated and experimentally validated human gene set generated by the Sanger Havana group (3) as part of the Gencode consortium (23).”
Gene prediction in novel fungal genomes using an ab initio algorithm with unsupervised training. (2008, background): “Reliable ab initio gene prediction in eukaryotic genomic sequences remains an open problem in spite of impressive progress made in developing gene prediction algorithms (Burge and Karlin 1997; Krogh 1997; Parra et al. 2000; Reese et al. 2000; Stanke and Waack 2003; Guigo et al. 2006).”
Using native and syntenically mapped cDNA alignments to improve de novo gene finding (2008, background): “There are a number of dual or multiple genome-based gene finders which use unannotated related genome sequences to predict the genes in one or two of these sequences by exploiting that functional regions are evolutionary better conserved (Meyer and Durbin (2002); Gross and Brent (2005) ).”
BRAKER2: automatic eukaryotic genome annotation with GeneMark-EP+ and AUGUSTUS supported by a protein database (2020, neutral): “Onaparallel avenue, yet another algorithm, AUGUS-TUS (9–14), was demonstrated to be one of the most accurate gene prediction tools (15–17).”