A large-scale evaluation of computational protein function prediction

By Predrag Radivojac, Wyatt Travis Clark, T. Oron, Alexandra M. Schnoes, Tobias Wittkop, Kiley S. Graim, Christopher S. Funk, Karin M. Verspoor, Asa Ben-Hur, G. Pandey and 91 more. 2013

1,002 citations recorded by the source corpus.

Automated annotation of protein function is challenging. As the number of sequenced genomes rapidly grows, the overwhelming majority of protein products can only be annotated computationally. If computational predictions are to be relied upon, it is crucial that the accuracy of these methods be high. Here we report the results from the first large-scale community-based critical assessment of protein function annotation (CAFA) experiment. Fifty-four methods representing the state of the art for protein function prediction were evaluated on a target set of 866 proteins from 11 organisms. Two findings stand out: (i) today's best protein function prediction algorithms substantially outperform widely used first-generation methods, with large gains on all types of targets; and (ii) although the top methods perform well enough to guide experiments, there is considerable need for improvement of currently available tools.

Subfields: Molecular Biology, Computational Biology & Bioinformatics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Molecular Biology

In Computational Biology & Bioinformatics

Citations in the corpus

Predicting multicellular function through multi-layer tissue networks (2017, methodology): “…information from protein interaction networks lack tissue speciﬁcity as they assume that cellular function is constant across organs and tissues (Barutcuoglu et al. , 2006; Mostafavi et al. , 2008; Radivojac et al. , 2013; Stojanova et al. , 2013; Kramer et al. , 2014; Zitnik and Zupan, 2015).”
UDSMProt: universal deep sequence models for protein classification (2019, methodology): “This is also the considered case of DeeProtein [15], with data from the CAFA3 challenge [28], which is a major attempt to establish a suitable evaluation framework for protein function prediction.”
Protein Secondary Structure Prediction Using Deep Convolutional Neural Fields (2015, neutral): “de as follows. 𝐸𝑖 𝑘(�)=𝑔(𝐻 𝑖 𝑘(�))∗∑[𝐸 � 𝑖(�)∗��,�] , if 𝑘=� () where 𝐸𝑖(�)=[𝛿(�𝑖=�)]−𝐸�𝑃𝑃(�|�,�,�,�)[𝛿(�𝑖=�)] () 𝐸𝑖 𝑘(�)=𝑔(𝐻 𝑖 𝑘(�))∗∑ ∑ [𝐸 𝑖−� 𝑘+1(�′)∗� � Æ𝑘+1 𝑘(�′,�)] �′=1 Ç𝑘 �=−Ç𝑘 , if 𝑘<� (8) Where 𝑔() (is the derivative of the activation function; it is �”
MRFalign: Protein Homology Detection through Alignment of Markov Random Fields (2014, background): “Sequence-based protein alignment and homology detection has been extensively studied and widely applied to many biological problems such as homology modeling [1-4], phylogeny inference [5-7] and protein function prediction [8-10].”
HIPPIE v2.0: enhancing meaningfulness and reliability of protein–protein interaction networks (2016, background): “Many studies use these PPI networks to identify commonalities among a group of proteins (3), to infer novel functions for a gene of interest (4) or for gene-disease association prediction (5,6).”
GOLabeler: Improving Sequence-based Large-scale Protein Function Prediction by Learning to Rank (2017, positive): “It is reported that using the similarity score (bit-score) between similar proteins and the query slightly improves the performance of just using the sequence identity [25].”
The Gene Ontology knowledgebase in 2023 (2023, neutral): “UniProt* (UniProt: the universal protein knowledgebase 2017) Human and also a wide variety of organisms not covered by other GOC members 185,121 30,927 MGI* (Bult et al. 2019) Mouse 106,435 8,051 Reactome* (Fabregat et al. 2018) Human pathways 92,178 6 TAIR* (Lamesch et al. 2012) A. thaliana (model ”
Exploiting ontology graph for predicting sparsely annotated gene function (2015, background): “…(HC) instead of binary classification, but its predictive performance for sparsely annotated functional labels is far from satisfactory (Clark and Radivojac, 2013; Eisner et al ., 2005; Guan et al ., 2008; Jiang et al ., 2014; Kim et al ., 2008; Obozinski et al ., 2008; Sokolov and Ben-Hur,…”
FFPred 3: feature-based function prediction for all Gene Ontology domains (2016, methodology): “Prediction accuracy was measured separately for each GO domain by precision-recall analysis as in similar studies following the lead of the CAFA experiments 34,35 .”
Leveraging implicit knowledge in neural networks for functional dissection and engineering of proteins (2019, methodology): “Using a Critical Assessment of Functional Annotation 3 (CAFA3) benchmarking dataset, we first show that DeeProtein reaches state-of-the-art performance 1,13 .”
Information-theoretic evaluation of predicted ontological annotations (2013, methodology): “In addition to knowledge representation, GO has also facilitated large-scale analyses and automated annotation of gene product function (Radivojac et al. , 2013).”
Panoramic view of a superfamily of phosphatases through substrate profiling (2015, methodology): “5 were annotated as follows: (1) eukaryotic polynucleotide kinase/phosphatases involved in DNA repair, (2) GmhB enzymes that prefer the alpha anomer of D-D-heptose-1,7-bisphosphate, (3) GmhB enzymes that take either the alpha or beta anomer, (4) bifunctional HisB enzymes that show specificity for hi”