A Survey of Indexing Techniques for Scalable Record Linkage and Deduplication

By Peter Christen. 2012

761 citations recorded by the source corpus.

Subfields: Computer Networks and Communications, Information Systems, Management Science and Operations Research, Information Retrieval & Search, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Information Systems

In Management Science and Operations Research

In Information Retrieval & Search

In Evaluation & Benchmarks

Citations in the corpus

SIGMa: simple greedy matching for aligning large knowledge bases (2012, methodology): “S0 has the purpose to increase the possible exploration of the graph when another strong source of information (which is not from the graph) can be used, and corresponds to the standard blocking or indexing heuristics in record linkage [8].”
Distributed Representations of Tuples for Entity Resolution (2017, methodology): “Blocking has been extensively studied as a way to scale ER systems and a good overview can be found in surveys such as [4, 11].”
NADEEF: a commodity data cleaning system (2013, neutral): “, [7, 25]).”
CrowdER: Crowdsourcing Entity Resolution (2012, methodology): “Figure 2(a) shows the remaining ten pairs 1 In practice, we can adopt some indexing techniques such as blocking and Q-gram based indexing [7] to avoid all-pairs comparison.”
A sample-and-clean framework for fast and accurate query processing on dirty data (2014, background): “A recent survey on duplicate detection has argued that the matching phase is typically much more expensive than the blocking phase [13].”
Efficient private record linkage of very large datasets (2013, methodology): “The presentation here will be limited to suitable candidates from the set of best performing methods in the comparison study of Christen (2012).”
A taxonomy of privacy-preserving record linkage techniques (2013, methodology): “To overcome the issues with data that are of low quality, q-gram based indexing techniques can be used that insert each record into several blocks by generating variations of the record’s blocking key value through the use of q-grams (sub-strings of length q characters) [17,19].”
Efficient two-party private blocking based on sorted nearest neighborhood clustering (2013, methodology): “RR is the fraction of record pairs that are removed by a blocking solution and PC is the fraction of true matching record pairs that are included in the candidate record pairs generated by a blocking solution [5].”
Visualizing Dataflow Graphs of Deep Learning Models in TensorFlow (2018, methodology): “We detect similar group nodes using a two-step approach inspired by the blocking technique for duplication detection in databases [17].”
Deep entity matching with pre-trained language models (2020, background): “EM solutions have tackled the blocking problem [2, 6, 14, 28, 45] and the matching problem with rules [9, 13, 38, 44], crowdsourcing [16, 18, 43], or machine learning [37, 8, 3, 16, 20].”
Data Matching (2012, methodology): “Data-Centric Systems and Applications Series Editors M. J. Carey S. Ceri Editorial Board P. Bernstein U. Dayal C. Faloutsos J. C. Freytag G. Gardarin W. Jonker V. Krishnamurthy M.-A. Neimat P. Valduriez G. Weikum K.-Y. Whang J. Widom For further volumes: http://www.springer.com/series/5258 Peter Chr”
Entity Linking with a Knowledge Base: Issues, Techniques, and Solutions (2015, background): “Another related problem is record linkage [34], [35], [36], [37], [38], [39], [40], [41] (also called duplicate detection, entity matching, and reference reconciliation) in the database community.”