High-dimensional cluster analysis with the Masked EM Algorithm

By Shabnam N. Kadir, Dan F. M. Goodman, Kenneth D. Harris. Neural Computation, 2013

307 citations recorded by the source corpus.

Abstract Cluster analysis faces two problems in high dimensions: the “curse of dimensionality” that can lead to overfitting and poor generalization performance and the sheer time taken for conventional algorithms to process large amounts of high-dimensional data. We describe a solution to these problems, designed for the application of spike sorting for next-generation, high-channel-count neural probes. In this problem, only a small subset of features provides information about the cluster membership of any one data vector, but this informative feature subset is not the same for all data points, rendering classical feature selection ineffective. We introduce a “masked EM” algorithm that allows accurate and time-efficient clustering of up to millions of points in thousands of dimensions. We demonstrate its applicability to synthetic data and to real-world high-channel-count spike sorting data.

Subfields: Artificial Intelligence, Cognitive Neuroscience, Signal Processing

View Paper: https://www.semanticscholar.org/paper/c3a237575f85789fa40e83e4d6cf01cb65da7ef4

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Cognitive Neuroscience

In Signal Processing

Citations in the corpus

Spike sorting for large, dense electrode arrays (2015, methodology): “To ensure the poor performance of the classical EM algorithm did not simply reflect incorrect parameter choice, we reran it for multiple values of the penalty parameter (which determines the number of clusters found 30 ), but this could not improve Classical EM performance.”
Kilosort: realtime spike-sorting for extracellular electrophysiology with hundreds of channels (2016, methodology): “When applied to such data, algorithms designed for tens of recording sites 18,19 suffer from substantial limitations.”
A spike sorting toolbox for up to thousands of electrodes validated with ground truth recordings in vitro and in vivo (2018, methodology): “Then, we randomly selected some of those templates wj and shuffled the list of their electrodes before injecting them elsewhere in the datasets at controlled firing rates (Harris et al., 2000; Rossant et al., 2016; Kadir et al., 2014; Segev et al., 2004; Marre et al., 2012; Chung et al., 2017).”