Evaluating Virtual Screening Methods: Good and Bad Metrics for the "Early Recognition" Problem

By Jean-François Truchon, C. Bayly. 2007

813 citations recorded by the source corpus.

Subfields: Artificial Intelligence, General Health Professions, Molecular Biology

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In General Health Professions

In Molecular Biology

Citations in the corpus

What do we know and when do we know it? (2008, methodology): “It is against this backdrop that metrics such as RIE [28] and BedROC [ 29 ] were developed.”
Selectivity by Small-Molecule Inhibitors of Protein Interactions Can Be Driven by Protein Surface Fluctuations (2015, neutral): “α=20 (183), a key measure designed specifically to assess performance for “early recognition”, is 0.”
Comparing structural fingerprints using a literature-based similarity benchmark (2016, methodology): “A particular problem is the continued use of metrics related to the average rank of the actives and that, as a result, cannot even distinguish between trivial examples of methods with good versus poor performance [36].”
Towards the development of universal, fast and highly accurate docking/scoring methods: a long way to go (2008, background): “Additionally, the metrics for success in virtual screening (usually enrichment factors and ROC curves) are sometimes difficult to interpret (Truchon and Bayly, 2007).”
Docking and scoring with ICM: the benchmarking results and strategies for improvement (2012, background): “Because the top scoring compounds will most likely be selected for further testing, early enrichment measures are important to identify successful VLS runs where active compounds are ranked ahead of decoys on a large score list [16].”
Structure-Based Virtual Screening: From Classical to Artificial Intelligence (2020, background): “…square deviation (RMSD) (Hawkins et al., 2008), receiver operating characteristic (ROC), area under the curve ROC (AUC-ROC) (Flach and Wu, 2005; Trott and Olson, 2009) enrichment factors (EFs) (Truchon and Bayly, 2007) and Boltzmann-enhanced discrimination of ROC (BEDROC) (Truchon and Bayly, 2007).”
Molecular Shape and Medicinal Chemistry: A Perspective (2010, methodology): “Simulation studies in MRL by Truchon and Bayly 25 also reinforced the need for more actives.”
Open-source platform to benchmark fingerprints for ligand-based virtual screening (2013, methodology): “Truchon and Bayly [9] have analytically calculated the random average and obtained an exact formula, RIE(α) = N n ∑n i=1 e−αri/N 1−e−α eα/N −1 .”
Large-Scale Systematic Analysis of 2D Fingerprint Methods and Parameters to Improve Virtual Screening Enrichments (2010, methodology): “%) and BEDROC(R ) 160.9) for the different fingerprint types across the 11 targets are shown in Table 7.”