High Dimensional Classification Using Features Annealed Independence Rules.

By Jianqing Fan, Yingying Fan. Annals of Statistics, 2007

574 citations recorded by the source corpus.

Classification using high-dimensional features arises frequently in many contemporary statistical studies such as tumor classification using microarray or other high-throughput data. The impact of dimensionality on classifications is largely poorly understood. In a seminal paper, Bickel and Levina (2004) show that the Fisher discriminant performs poorly due to diverging spectra and they propose to use the independence rule to overcome the problem. We first demonstrate that even for the independence classification rule, classification using all the features can be as bad as the random guessing due to noise accumulation in estimating population centroids in high-dimensional feature space. In fact, we demonstrate further that almost all linear discriminants can perform as bad as the random guessing. Thus, it is paramountly important to select a subset of important features for high-dimensional classification, resulting in Features Annealed Independence Rules (FAIR). The conditions under which all the important features can be selected by the two-sample t-statistic are established. The choice of the optimal number of features, or equivalently, the threshold value of the test statistics are proposed based on an upper bound of the classification error. Simulation studies and real data analysis support our theoretical results and demonstrate convincingly the advantage of our new classification procedure.

Subfields: Analytical Chemistry, Artificial Intelligence, Molecular Biology

View Paper: https://www.semanticscholar.org/paper/0eb7d303011f6f879a0bcb118703003dca49a4d0

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Analytical Chemistry

In Artificial Intelligence

In Molecular Biology

Citations in the corpus

Sure independence screening for ultrahigh dimensional feature space (2006, methodology): “See Fan and Fan (2008) for a sure screening property.”
Sure independence screening in generalized linear models with NP-dimensionality (2009, background): “and other folded-concave penalty in Fan and Li (2001), the Dantzig selector in Candes and Tao (2007) and their related methods (Zou, 2006; Zou and Li, 2008).”
Penalized classification using Fisher’s linear discriminant (2011, background): “…years, a number of authors have proposed extensions of LDA to the high-dimensional setting in order to achieve sparsity (Tibshirani et al., 2002, 2003; Guo et al., 2007; Trendaﬁlov and Jolliﬀe, 2007; Grosenick et al., 2008; Leng, 2008; Fan and Fan, 2008; Shao et al., 2011; Clemmensen et al., 2011).”
On the Use of Reproducing Kernel Hilbert Spaces in Functional Classification (2015, methodology): “• The linear SVM has been performed with the MATLAB version of the LIBLINEAR library (see Fan et al .”
Two-Stage Procedures for High-Dimensional Data (2011, background): “Fan and Fan (2008), Meinshausen et al. (2009), and Wasserman and Roeder (2009) considered this type of problem.”
Asymptotic Equivalence of Regularization Methods in Thresholded Parameter Space (2013, methodology): “This data set, which was also analyzed in Fan and Fan (2008), consists of 136 patient samples with 77 from the prostate tumor group (labeled as 1) and 59 from the normal group (labeled as 0). For each patient, we have the gene expression measurements for 12,600 genes. Following Singh et al. (2002) a”
Computer-assisted recognition of dolphin individuals using dorsal fin pigmentations (2016, background): “…class centroids and the group centroid is considered to be a natural criterion to determine which features should be selected for a classiﬁer [18] – the cat-score is an extension that generalizes the t -score to account for correlation between features and is therefore suitable to rank…”
Sequential sufficient dimension reduction for large p, small n problems (2015, methodology): “This data set was first analysed by Golub et al. (1999), and later by Chiaromonte and Martinelli (2002), Dudoit et al. (2002) and Fan and Fan (2008), among others.”
Challenges of Big Data Analysis (2013, unknown): “…us to better understand heterogeneity, shedding light toward studies such as exploring the association be-tween certain covariates (e.g. genes or SNPs) and rare outcomes (e.g. rare diseases or diseases in small populations) and understanding why certain treatments (e.g. chemotherapy) benefit a…”
Feature Selection in High-Dimensional Classification (2013, background): “More recently, numerous alternative approaches have been proposed by taking more complex covariance matrix structures into consideration (Fan et al., 2010; Shao et al., 2011; Cai & Liu, 2011; Mai et al., 2012).”
High dimensional ordinary least squares projection for screening variables (2015, methodology): “The screening operation has been extended, for example, to generalized linear models (Fan and Fan, 2008; Fan, et al., 2009; Fan and Song, 2010), additive models (Fan, et al., 2011), hazard regression (Zhao and Li, 2012; Gorst-Rasmussen and Scheike, 2013), and to accommodate conditional correlation…”
Random matrix theory in statistics: A review (2014, background): “A different approach that ignores the covariance structure, but assumes sparsity of μ1 μ2, has been proposed by Fan and Fan (2008).”
A Selective Overview of Variable Selection in High Dimensional Feature Space (2009, methodology): “Fan and Fan (2008) prove that when dimensionality p grows no faster than the exponential rate of the sample size, if the lowest signal level is not too small, the two-sample t test can select all important features with probability tending to 1.”
Using Generalized Correlation to Effect Variable Selection in Very High Dimensional Problems (2009, result): “There is considerable overlap between the genes we found using this approach, and those discovered in other studies (Golub et al. 1999; Tibshirani et al. 2002; Hall, Titterington, and Xue 2007b; Fan and Fan 2008; Fan and Lv 2008).”
Independence test for high dimensional data based on regularized canonical correlation coefficients (2015, background): “There have been a substantial body of research work dealing with high dimensional data, for example, [2, 4, 7, 9, 10, 12], etc.”