A Comparison of Fast Blocking Methods for Record Linkage

By Rohan A. Baxter, Peter Christen, Tim Churches. Knowledge Discovery and Data Mining, 2003

427 citations recorded by the source corpus.

View Paper: https://www.semanticscholar.org/paper/761b151368e709b972a1fa2c6f672ad6c675f37a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Data Matching (2012, methodology): “A major result of this and other comparative studies [20, 64] is that an important factor for successful indexing is the appropriated definition of blocking or sorting keys.”
Duplicate Record Detection: A Survey (2007, background): “Baxter et al. [97] perform an experimental comparison of canopy-based approaches with traditional blocking and show that the flexible nature of canopies can significantly improve the quality and speed of duplicate detection.”
A Survey of Indexing Techniques for Scalable Record Linkage and Deduplication (2012, methodology): “Earlier surveys have compared four or less indexing techniques only [32], [33].”
An Introduction to Duplicate Detection (2010, methodology): “1 BLOCKING Blocking methods pursue the simple idea of partitioning the set of records into disjoint partitions (blocks) and then comparing all pairs of records only within each block [Ananthakrishna et al., 2002; Baxter et al., 2003; Bilenko et al., 2006].”
Frameworks for entity matching: A comparison (2010, methodology): “Overlapping methods may result in overlapping blocks of entities; implementations include the (multi-pass) sorted neighborhood approach [33], bi-gram indexing [4], canopy clustering [40] and iterative blocking [59].”
Data Curation at Scale: The Data Tamer System (2013, background): “, [7, 14]) currently used in entity resolution in large datasets.”
Privacy-Preserving Data Linkage and Geocoding: Current Approaches and Research Directions (2006, unknown): “Techniques known as blocking [ 4 , 15] are applied to reduce the number of record pairs that will be compared.”
Large-Scale Collective Entity Matching (2011, methodology): “Total covering is a natural extension of the notion of blocking, used extensively in entity matching [20, 15].”
Distributed Representations of Tuples for Entity Resolution (2017, unknown): “Blocking has been extensively studied as a way to scale ER systems and a good overview can be found in surveys such as [4, 11].”
Adaptive Blocking: Learning to Scale Up Record Linkage (2006, background): “A number of blocking algorithms have been proposed by researchers in recent years [12 , 18, 15, 24, 1, 6, 19, 37].”
A taxonomy of privacy-preserving record linkage techniques (2013, methodology): “To overcome the issues with data that are of low quality, q-gram based indexing techniques can be used that insert each record into several blocks by generating variations of the record’s blocking key value through the use of q-grams (sub-strings of length q characters) [17,19].”
Entity resolution with iterative blocking (2009, methodology): “Another line of research is blocking techniques that focus on ER performance [2].”
Deep entity matching with pre-trained language models (2020, background): “EM solutions have tackled the blocking problem [2, 6, 14, 28, 45] and the matching problem with rules [9, 13, 38, 44], crowdsourcing [16, 18, 43], or machine learning [37, 8, 3, 16, 20].”
Beyond data warehousing: what's next in business intelligence? (2004, background): “purge/merge problem [2] and duplicate detection [13]) rely on the presence of a materialized integrated level; we expect”
Evaluation of entity resolution approaches on real-world match problems (2010, background): “, evaluation of different string similarity metrics [11] and of blocking approaches [3].”
Febrl: a freely available record linkage system with a graphical user interface (2008, methodology): “‘QGramIndex’, which uses sub-strings of length q (for example bigrams, where q = 2) to allow fuzzy blocking (Baxter et al. 2003); ‘CanopyIndex’, which employs overlapping canopy clustering using TF-IDF or Jaccard similarity (Cohen and Richman 2002);”