Estimating mutual information using B-spline functions – an improved similarity measure for analysing gene expression data

By Carsten O. Daub, Ralph E. Steuer, Joachim Selbig, Sebastian Kloska. BMC Bioinformatics, 2004

312 citations recorded by the source corpus.

BackgroundThe information theoretic concept of mutual information provides a general framework to evaluate dependencies between variables. In the context of the clustering of genes with similar patterns of expression it has been suggested as a general quantity of similarity to extend commonly used linear measures. Since mutual information is defined in terms of discrete variables, its application to continuous data requires the use of binning procedures, which can lead to significant numerical errors for datasets of small or moderate size.ResultsIn this work, we propose a method for the numerical estimation of mutual information from continuous data. We investigate the characteristic properties arising from the application of our algorithm and show that our approach outperforms commonly used algorithms: The significance, as a measure of the power of distinction from random correlation, is significantly increased. This concept is subsequently illustrated on two large-scale gene expression datasets and the results are compared to those obtained using other similarity measures.A C++ source code of our algorithm is available for non-commercial use from kloska@scienion.de upon request.ConclusionThe utilisation of mutual information as similarity measure enables the detection of non-linear correlations in gene expression datasets. Frequently applied linear correlation measures, which are often used on an ad-hoc basis without further justification, are thereby extended.

Subfields: Molecular Biology, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/46c37a4dccdbf9e906c4cb39a5d91647ae4f84ac

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Molecular Biology

In Statistics and Probability

Citations in the corpus

Causality detection based on information-theoretic approaches in time series analysis (2007, methodology): “Daub et al. [56] gave numerical estimates of bias and variance with dataset size of N for the estimator I M,k (X ; Y) for statistically independent datasets and for k = 3.”
minet: A R/Bioconductor Package for Inferring Large Transcriptional Networks Using Mutual Information (2008, background): “A fast entropy estimation is therefore essential for an effective network inference based on MI. Entropy estimation has gained much interest in feature selection and network inference over the last decade [22].”
Maximum Likelihood Estimation of Functionals of Discrete Distributions (2014, methodology): “Among them we mention the Miller–Madow bias-corrected estimator and its variants [29]–[31], the jackknife estimator [32], the shrinkage estimator [33], the coverage adjusted estimator [34], the Best Upper Bound (BUB) estimator [14], the B-Splines estimator [35], and [36], [37] etc.”
Equitability, mutual information, and the maximal information coefficient (2013, background): “m sparse data is a nontrivial problem [7, 11]. Indeed, many approaches for estimating mutual information (or entropy, a closely related quantity) have been proposed; a non-comprehensive list includes [3, 6, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24]. While comparisons of various estimation ”
An information theoretic alternative to model a natural system using observational information alone (2014, background): “MI is widely utilized in diverse disciplines, which include bioinformatics [Schena et al., 1995; Velculescu et al., 1995; D’haeseleer et al., 2000; Carsten et al., 2004; Hahn et al., 2005], physics [Fraser and Swinney, 1986], image recognition [Th enevaz and Unser, 2000], speech recognition [Ellis…”
Large-Scale Mapping and Validation of Escherichia coli Transcriptional Regulation from a Compendium of Expression Profiles (2007, methodology): “To compute mutual information, we used a B-spline smoothing and discretization method, except for comparison testing with the original algorithms, as noted below [47].”
Comparison of co-expression measures: mutual information, correlation, and model based indices (2012, background): “But, several strategies have been proposed in the literature [7,28,29].”
Minimax Estimation of Functionals of Discrete Distributions (2014, methodology): “If we trace the progress on entropy estimation in the non-asymptotic regime, we find several thrusts in various communities, including, for example, the Miller–Madow bias-corrected estimator and its variants [21]–[23], the jackknified estimator [24], the shrinkage estimator [25], the Bayes estimator”