Comparison of multivariate classifiers and response normalizations for pattern-information fMRI

By Masaya Misaki, Youn Kim, Peter A. Bandettini, Nikolaus Kriegeskorte. NeuroImage, 2010

485 citations recorded by the source corpus.

A popular method for investigating whether stimulus information is present in fMRI response patterns is to attempt to “decode” the stimuli from the response patterns with a multivariate classifier. The sensitivity for detecting the information depends on the particular classifier used. However, little is known about the relative performance of different classifiers on fMRI data. Here we compared six multivariate classifiers and investigated how the response-amplitude estimate used (beta or t-value) and different pattern normalizations affect classification performance. The compared classifiers were a pattern-correlation classifier, a k-nearest-neighbors classifier, Fisher’s linear discriminant, Gaussian naïve Bayes, and linear and nonlinear (radial-basis-function-kernel) support vector machines. We compared these classifiers’ accuracy at decoding the category of visual objects from response patterns in human early visual and inferior temporal cortex acquired in an event-related design with BOLD fMRI at 3T using SENSE and isotropic voxels of about 2-mm width. Overall, Fisher’s linear discriminant (with an optimal-shrinkage covariance estimator) and the linear support vector machine performed best. The pattern-correlation classifier often performed similarly as those two classifiers. The nonlinear classifiers never performed better and sometimes significantly worse than the linear classifiers, suggesting overfitting. Defining response patterns by t-values (or in error-standard-deviation units) rather than by beta estimates (in % signal change) to define the patterns appeared advantageous. Cross-validation by a leave-one-stimulus-pair-out method gave higher accuracies than a leave-one-run-out method, suggesting that generalization to independent runs (which more safely ensures independence of the test set) is more challenging than generalization to novel stimuli within the same category. Independent selection of fewer more visually responsive voxels tended to yield better decoding performance for all classifiers. Normalizing mean and standard deviation of the response patterns either across stimuli or across voxels had no significant effect on decoding performance. Overall our results suggest that linear decoders based on t-value patterns may perform best in the present scenario of visual object representations measured for about 60-minutes per subject with 3T fMRI.

Subfields: Cognitive Neuroscience

View Paper: https://www.semanticscholar.org/paper/13efe7c7d3107d7bbecb7660990438eef288ad80

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

Citations in the corpus

Deconvolving BOLD activation in event-related designs for multivoxel pattern classification analyses (2011, methodology): “Misaki et al. (2010) found that when using the LS-A approach for obtaining activation patterns, it was beneficial to use the t-statistics for the activation estimate rather than the raw parameter estimates.”
On the interpretation of weight vectors of linear models in multivariate neuroimaging (2014, background): “Moreover, they typically yield comparable estimation accuracy in many applications (Misaki et al., 2010).”
Interpretable LSTMs For Whole-Brain Neuroimaging Analyses (2018, background): “Although it has been shown that linear methods are often advantageous when decoding cognitive states of an individual from small clusters of brain activity only [5], it is an open question whether this is also true when larger clusters of brain activity are used (i.”
Reliability of dissimilarity measures for multi-voxel pattern analysis (2016, unknown): “The same aim is achieved by using t values instead of regression estimates, which has been shown to increase classification performance of linear support vector machines (Misaki et al., 2010).”
Visualization of Nonlinear Classification Models in Neuroimaging - Signed Sensitivity Maps (2012, background): “For additional discussion of nonlinear classification in neuroimaging, see (Mørch et al., 1997; Cox and Savoy, 2003; LaConte et al., 2005; Haynes and Rees, 2006; Pereira et al., 2009; Misaki et al., 2010; Schmah et al., 2010; Rasmussen et al., 2011).”
Decoding Dynamic Brain Patterns from Evoked Responses: A Tutorial on Multivariate Pattern Analysis Applied to Time Series Neuroimaging Data (2016, background): “Therefore, for brain decoding studies, linear classifiers are generally preferred, as they are simpler in nature, making interpretation less complex (Schwarzkopf & Rees, 2011; Misaki et al., 2010; Müller et al., 2003).”
The Decoding Toolbox (TDT): a versatile software package for multivariate analyses of functional imaging data (2015, background): “Multiple classiﬁers have been compared across different data sets, with variable results (Cox and Savoy, 2003; Pereira et al., 2009; Misaki et al., 2010).”
Less Is More: Expectation Sharpens Representations in the Primary Visual Cortex (2012, unknown): “variance) obtained for each voxel comprised the data for further analysis (Misaki et al., 2010).”