Robust and efficient fuzzy match for online data cleaning

By Surajit Chaudhuri, Kris Ganjam, Venkatesh Ganti, Rajeev Motwani. ACM SIGMOD Conference, 2003

568 citations recorded by the source corpus.

Subfields: Computer Networks and Communications, Management Science and Operations Research, Signal Processing

View Paper: https://www.semanticscholar.org/paper/1c3c1ce30db5a50a51b3ca8eb9448d763aba226a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Management Science and Operations Research

In Signal Processing

Citations in the corpus

A Latent Dirichlet Model for Unsupervised Entity Resolution (2005, background): “Non-probabilistic approaches that take relational features into account for data integration have been proposed [11, 7, 1, 3, 20, 12].”
Reference reconciliation in complex information spaces (2005, methodology): “Finally, several work [21, 27, 6, 22] has addressed the computational cost of reference reconciliation.”
InfoGather: entity augmentation and attribute discovery by holistic matching with web tables (2012, methodology): “We then perform fuzzy grouping [7] on the x i ’s to get the groups G q = { g 1 , g 2 , . . . } , such that, ∀ x i ∈ g k , x i ≈ v k , where v k is the centroid or the representative of group g k .”
Large-Scale Deduplication with Constraints Using Dedupalog (2009, methodology): “This includes work on basic string similarity measures such as edit distance and variations [33], [34], Jaccard, TF/IDF [3], [35], and Jaro [36], and more complex similarity measures at the record or multi-attribute level [33], [37], [38], which are typically derived by combining single attribute si”
Efficient exact set-similarity joins (2006, methodology): “A large number of different similarity functions such as edit distance, cosine similarity, jaccard similarity, and generalized edit distance [5] have been traditionally used in similarity joins.”
Duplicate Record Detection: A Survey (2007, background): “For example, computing the edit distance between two long strings σ1 and σ2, respectively, has a cost of O(|σ1| · |σ2|); just checking if they are within a prespeci ed edit distance threshold k can reduce the complexity to O(max{|σ1|, |σ2”
Privacy-Preserving Data Linkage and Geocoding: Current Approaches and Research Directions (2006, methodology): “Many linkage methods based on machine learning, artificial intelligence and information retrieval techniques have been developed in the past few years [6, 10 , 11, 20, 21, 22, 25, 27, 34, 35, 38, 41, 46, 50, 53].”
Efficient Merging and Filtering Algorithms for Approximate String Searches (2008, methodology): “, [3], [4]) have been proposed for answering approximate string queries efficiently.”
Management of probabilistic data: foundations and challenges (2007, background): “This is a central problem in data cleaning and data integration, and has also been called record linkage, deduplication, or merge-purge [34, 3, 13, 37, 43, 49, 15, 80, 7].”
Adaptive Blocking: Learning to Scale Up Record Linkage (2006, background): “, edit distance [6, 19] or Jaccard similarity [24], while others assume that forming blocks based on lexicographic sorting of records on some field(s) yields an optimal strategy [15, 37].”
Deduplication of Scholarly Documents using Locality Sensitive Hashing and Word Embeddings (2020, background): “(Chaudhuri et al., 2003), (Jiang et al.”
A Primitive Operator for Similarity Joins in Data Cleaning (2006, background): “Previous implementations for the generalized edit similarity were probabilistic [4]; hence, we do not compare our implementations with the earlier strategy.”
Efficient set joins on similarity predicates (2004, background): “…the intersect size to union size is greater than a fraction f. Weighted match > T where elements are attached with arbitrary weights (for example, inverse of their frequency in the database) and the goal is to .nd all set pairs where the total weight of the common/overlapping elements is > T .”
Enterprise Data Analysis and Visualization: An Interview Study (2012, background): “Despite much research from the database and statistics communities [4, 8, 16, 19, 31, 32], little visualization research addresses discovery, wrangling or profiling challenges.”