Clustering by pattern similarity in large data sets

By Haixun Wang, Wei Wang, Jiong Yang, Philip S. Yu. ACM SIGMOD Conference, 2002

581 citations recorded by the source corpus.

Clustering is the process of grouping a set of objects into classes of similar objects. Although definitions of similarity vary from one clustering model to another, in most of these models the concept of similarity is based on distances, e.g., Euclidean distance or cosine distance. In other words, similar objects are required to have close values on at least a set of dimensions. In this paper, we explore a more general type of similarity. Under the pCluster model we proposed, two objects are similar if they exhibit a coherent pattern on a subset of dimensions. For instance, in DNA microarray analysis, the expression levels of two genes may rise and fall synchronously in response to a set of environmental stimuli. Although the magnitude of their expression levels may not be close, the patterns they exhibit can be very much alike. Discovery of such clusters of genes is essential in revealing significant connections in gene regulatory networks. E-commerce applications, such as collaborative filtering, can also benefit from the new model, which captures not only the closeness of values of certain leading indicators but also the closeness of (purchasing, browsing, etc.) patterns exhibited by the customers. Our paper introduces an effective algorithm to detect such clusters, and we perform tests on several real and synthetic data sets to show its effectiveness.

Subfields: Artificial Intelligence, Information Systems, Statistical and Nonlinear Physics

View Paper: https://www.semanticscholar.org/paper/0234b679d898f24e8113b5eb8ef50104a9dd21f8

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Statistical and Nonlinear Physics

Citations in the corpus

A review on time series data mining (2011, methodology): “For the problem of time series pattern discovery, a common group of techniques being employed is distance-based clustering (Das et al., 1998; Oates, 1999; Wang et al., 2002).”
Mining Frequent Patterns in Data Streams at Multiple Time Granularities (2002, background): “…Srikant1995], structured pattern mining [Kuramochi & Karypis2001], iceberg cube computation [Beyer & Ramakrishnan1999], cube gradient analysis [Imielinski , Khachiyan, & Abdulghani2002], associative classification [Liu, Hsu, & Ma1998], frequent pattern-based clustering [Wang et al.2002], and so on.”
A Comparison Study on Similarity and Dissimilarity Measures in Clustering Continuous Data (2015, methodology): “Although Euclidean distance is very common in clustering, it has a drawback: if two data vectors have no attribute values in common, they may have a smaller distance than the other pair of data vectors containing the same attribute values [31,35,36].”
Finding large average submatrices in high dimensional data (2009, unknown): “…equal [Friedman and Meulman (2004)]; the elements of U are well-fit by a 2-way ANOVA model [Cheng and Church (2000); Lazzeroni and Owen (2002); Wang et al. (2002)]; the rows of U have equal [Ben-Dor et al. (2003)] or approximately equal [Liu, Yang and Wang (2004)] rank statistics; and all…”
A review of clustering techniques and developments (2017, background): “Big data [161-168] is also an emerging issue.”
Biclustering algorithms for biological data analysis: a survey (2004, unknown): “Wang et al. [28] also used the MovieLens data.”
FABIA: factor analysis for bicluster acquisition (2010, background): “Received on December 23, 2009; revised on March 26, 2010; accepted on April 20, 2010”
Survey of clustering algorithms (2005, methodology): “A generalized subspace clustering model, pCluster was proposed in [279].”
Time-series clustering - A decade review (2015, background): “Clustering of time-series data is mostly utilized for discovery of interesting patterns in time-series datasets [27,28].”
Frequent pattern mining: current status and future directions (2007, methodology): “Condensed Cube (Wang et al. 2002) and Dwarf Cube (Sismanis et al.”
Biclustering in data mining (2008, methodology): “We refer the reader to papers [22,23,72–74] for details.”