Adaptive duplicate detection using learnable string similarity measures

By Mikhail Bilenko, Raymond J. Mooney. 2003

1,110 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Information Systems, Management Science and Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Management Science and Operations Research

Citations in the corpus

Fast Learning from Distributed Datasets without Entity Matching (2016, background): “On the other hand, learning-basedER [5] as well as manifold alignment techniques [15] are viable only knowing some ground truth matches — which are not required for working with rados.”
A Survey on Metric Learning for Feature Vectors and Structured Data (2013, methodology): “a joint distribution over the edit operations: X (u,v)∈(Σ∪{$})2\{$,$} Cuv +c(#) = 1, with c(#) >0 and Cuv ≥ 0, where # is a termination symbol and c(#) the associated cost (probability). Note that Bilenko and Mooney (2003) extended thisapproach to theNeedleman-Wunsch score with aﬃne gap penalty a”
Trends in Cleaning Relational Data: Consistency and Deduplication (2015, methodology): “Examples of classifier models include Naïve Bayes [122], decision tree [20], and support vector machine (SVM) [15].”
Learning the structure of Markov logic networks (2005, methodology): “…databases: the UW-CSE database used by Richardson and Domingos (2004) (available at http://www.cs.washington.edu/ai/mln), and McCallum's Cora database of computer science citations as segmented by Bilenko and Mooney (2003) (available at http://www.cs.utexas.edu/users/ml/riddle/data/cora.tar.gz).”
Large-Scale Deduplication with Constraints Using Dedupalog (2009, methodology): “This includes work on basic string similarity measures such as edit distance and variations [33], [34], Jaccard, TF/IDF [3], [35], and Jaro [36], and more complex similarity measures at the record or multi-attribute level [33], [37], [38], which are typically derived by combining single attribute si”
Query-Oriented Data Cleaning with Oracles (2015, methodology): “It is a powerful tool that has been employed for database cleaning tasks such as entity/conﬂict resolution [61, 65], duplicate detection [6, 11], schema matching [67, 35, 43], and ﬁlling up missing data [50, 49, 27].”
Learning similarity metrics for event identification in social media (2010, methodology): “In domains where more than one similarity metric is appropriate, several approaches have been proposed for combining multiple similarities using machine learning techniques [8, 10, 12, 13].”
Deep entity matching with pre-trained language models (2020, background): “EM solutions have tackled the blocking problem [2, 6, 14, 28, 45] and the matching problem with rules [9, 13, 38, 44], crowdsourcing [16, 18, 43], or machine learning [37, 8, 3, 16, 20].”
A Comparison of Personal Name Matching: Techniques and Practical Issues (2006, methodology): “This will also limit the use of trained name matching algorithms [2, 9, 31] that are adapted to deal with certain types of variations and errors.”
Joint Inference in Information Extraction (2007, methodology): “The Cora dataset was created by Andrew McCallum and later segmented by Bilenko & Mooney (2003) .”
Reference reconciliation in complex information spaces (2005, methodology): “In order to demonstrate the applicability of our algorithm in a more conventional setting, our second experiment uses the subset of the Cora dataset provided by McCallum [11] and used previously in [8, 3, 30].”
Joint Entity Resolution (2012, background): “[6] provides training for the specific domain of string similarity measures.”
Adaptive Blocking: Learning to Scale Up Record Linkage (2006, background): “Second, similarity is estimated for the candidate pairs using one or more similarity functions [11, 39, 35, 4].”
CrowdER: Crowdsourcing Entity Resolution (2012, positive): “Finally, SVM returned a ranked list of the remaining pairs sorted based on the likelihood given by the classiﬁer [4]. hybrid : Our hybrid human-machine workﬂow ﬁrst uses simjoin to obtain a set of pairs based on a speciﬁed threshold, and then veriﬁes these pairs by using the cluster-based HITs…”