Standardised Benchmarking in the Quest for Orthologs

By Adrian M. Altenhoff, Brigitte Boeckmann, Salvador Capella-Gutiérrez, Daniel A. Dalquen, Todd F. DeLuca, Kristoffer Forslund, Jaime Huerta-Cepas, Benjamin Linard, Cécile Pereira, Leszek P. Pryszcz and 20 more. 2016

213 citations recorded by the source corpus.

Achieving high accuracy in orthology inference is essential for many comparative, evolutionary and functional genomic analyses, yet the true evolutionary history of genes is generally unknown and orthologs are used for very different applications across phyla, requiring different precision–recall trade-offs. As a result, it is difficult to assess the performance of orthology inference methods. Here, we present a community effort to establish standards and an automated web-based service to facilitate orthology benchmarking. Using this service, we characterize 15 well-established inference methods and resources on a battery of 20 different benchmarks. Standardized benchmarking provides a way for users to identify the most effective methods for the problem at hand, sets a minimum requirement for new tools and resources, and guides the development of more accurate orthology inference methods.

Subfields: Molecular Biology, Safety Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Molecular Biology

Citations in the corpus

eggNOG 5.0: a hierarchical, functionally and phylogenetically annotated orthology resource based on 5090 organisms and 2502 viruses (2018, methodology): “ized benchmarking approaches (15) and reference datasets are developed and shared.”
UniProt: a worldwide hub of protein knowledge (2018, methodology): “UniProt reference proteomes are used by user communities such as the Quest for Orthologs (QfO) group as the ‘gold standard’ reference proteome dataset for ortho-logue benchmarking (8).”
OrthoVenn2: a web server for whole-genome comparison and annotation of orthologous clusters across multiple species (2019, methodology): “Finally, the accuracy of OrthoVenn2 was compared with other orthology inference methods using the benchmark service from the QfO community (29).”
Fast Genome-Wide Functional Annotation through Orthology Assignment by eggNOG-Mapper (2016, background): “Recent systematic evaluation (Altenhoff et al. 2016) revealed eggNOG (evolutionary genealogy of genes: Nonsupervised Orthologous Groups (OGs)) methodology to perform well in its task of distinguishing orthologous from paralogous gene groups.”
How to build phylogenetic species trees with OMA (2020, unknown): “In the Quest for Orthologs Benchmark 12 , the community benchmark for orthology inference, OMA Groups are consistently the most specific inference, although lacking in recall.”
Advances and Applications in the Quest for Orthologs (2019, methodology): “org; Last accessed April 2019 (Altenhoff et al. 2016), and here anyone can upload ortholog predictions that will be subjected to 20 different benchmarks.”
IUPred3: prediction of protein disorder enhanced with unambiguous experimental annotation and visualization of evolutionary conservation (2021, methodology): “First, orthologs were obtained by applying all-against-all GOPHER algorithm based prediction using protein sequences of the latest QFO (release 2020 04) reference dataset as the searching database (35,36).”
OMA standalone: orthology inference among public and custom genomes and transcriptomes (2018, methodology): “The OMA algorithm is known for having higher precision but lower recall than most other methods (Altenhoff et al., 2016) .”