Revealing strengths and weaknesses of methods for gene network inference

By Daniel Marbach, Robert J. Prill, Thomas Schaffter, Claudio Mattiussi, Dario Floreano, Gustavo A. Stolovitzky. Proceedings of the National Academy of Sciences of the United States of America, 2010

795 citations recorded by the source corpus.

Numerous methods have been developed for inferring gene regulatory networks from expression data, however, both their absolute and comparative performance remain poorly understood. In this paper, we introduce a framework for critical performance assessment of methods for gene network inference. We present an in silico benchmark suite that we provided as a blinded, community-wide challenge within the context of the DREAM (Dialogue on Reverse Engineering Assessment and Methods) project. We assess the performance of 29 gene-network-inference methods, which have been applied independently by participating teams. Performance profiling reveals that current inference methods are affected, to various degrees, by different types of systematic prediction errors. In particular, all but the best-performing method failed to accurately infer multiple regulatory inputs (combinatorial regulation) of genes. The results of this community-wide experiment show that reliable network inference from gene expression data remains an unsolved problem, and they indicate potential ways of network reconstruction improvements.

Subfields: Molecular Biology

View Paper: https://www.semanticscholar.org/paper/8a5329acdae6e2ec4795b2218840ca2d569b3e77

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Molecular Biology

Citations in the corpus

Inferring Regulatory Networks from Expression Data Using Tree-Based Methods (2010, result): “This suggests that the performance of GENIE3 at retrieving a regulator of one gene degrades as the number of regulators of this gene increases, as also observed from the analysis of the results of the DREAM3 challenge in [42].”
Structure and dynamics of molecular networks: A novel paradigm of drug discovery: A comprehensive review (2012, methodology): “…distinguishing between direct and indirect regulations; 3.) a better discrimination between causal relationships and coincidence; as well as 4.) network prediction in case of multiple regulatory inputs per node remain major challenges of the field (Tegnér & Bjorkegren, 2007; Marbach et al., 2010).”
Advantages and limitations of current network inference methods (2010, background): “In this regard, the DREAM (Dialogue on Reverse Engineering Assessments and Methods) initiativ”
Estimating the effect of joint interventions from observational data in sparse high-dimensional settings (2014, methodology): “All proofs, simulation results, and an illustration of our methods to the DREAM4 challenge [21] can be found in the Supplementary Material [24].”
Big Data Analytics in Healthcare (2015, methodology): “A tree-based method (using ensembles of regression trees) [174] and two-way ANOVA (analysis of variance) method [175] gave the highest performance in a recent DREAM challenge [160].”
RegNetwork: an integrated database of transcriptional and post-transcriptional regulatory networks in human and mouse (2015, background): “TFs and downstream genes (5)], the low accuracy of these existing methods due to the curse of dimensionality significantly limits their applications in practice (6).”
Wisdom of crowds for robust gene network inference (2012, methodology): “For improving individual inference approaches we suggest the following: (1) optimally exploit direct transcription factor perturbations; (2) employ strategies to avoid over-fitting, such as data resampling; (3) develop more effective approaches to distinguish direct from indirect regulation (feed-fo”
Characterization and greedy learning of interventional Markov equivalence classes of directed acyclic graphs (2011, methodology): “Applying GIES to synthetic data sets from the DREAM4 challenge (Marbach et al., 2010), we got better predictions of gene expression levels of knockout or knockdown experiments than with observational estimation methods.”
Data Based Identification and Prediction of Nonlinear and Complex Dynamical Systems (2016, background): “Based on a motif analysis, it was found [365] that three types of generic and systematic errors exist for inferring networks: (a) fan-out error, (b) fan-in error, and (c) cascade error.”
Gene Regulatory Network Inference from Single-Cell Data Using Multivariate Information Measures (2016, methodology): “3B), and it has become a standard tool for performance evaluation of network inference algorithms (Schaffter et al. 2011, Marbach et al. 2010, Marbach et al. 2012).”
Inference of complex biological networks: distinguishability issues and optimization-based solutions (2011, methodology): “gene expression [5–13], signal transduction [14–17] and metabolic networks [18–25].”