TAILOR: a record linkage toolbox

By Mohamed G. Elfeky, Ahmed K. Elmagarmid, Vassilios S. Verykios. Proceedings / International Conference on Data Engineering, 2002

347 citations recorded by the source corpus.

Data cleaning is a vital process that ensures the quality of data stored in real-world databases. Data cleaning problems are frequently encountered in many research areas, such as knowledge discovery in databases, data warehousing, system integration and e-services. The process of identifying the record pairs that represent the same entity (duplicate records), commonly known as record linkage, is one of the essential elements of data cleaning. In this paper, we address the record linkage problem by adopting a machine learning approach. Three models are proposed and are analyzed empirically. Since no existing model, including those proposed in this paper, has been proved to be superior, we have developed an interactive record linkage toolbox named TAILOR (backwards acronym for "RecOrd LInkAge Toolbox"). Users of TAILOR can build their own record linkage models by tuning system parameters and by plugging in in-house-developed and public-domain tools. The proposed toolbox serves as a framework for the record linkage process, and is designed in an extensible way to interface with existing and future record linkage models. We have conducted an extensive experimental study to evaluate our proposed models using not only synthetic but also real data. The results show that the proposed machine-learning record linkage models outperform the existing ones both in accuracy and in performance.

Subfields: Artificial Intelligence, Information Systems, Management Science and Operations Research

View Paper: https://www.semanticscholar.org/paper/ae3b641b7e8706e3cd60cc504a224af8959b1df6

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Management Science and Operations Research

Citations in the corpus

NADEEF: a commodity data cleaning system (2013, methodology): “AJAX [17] and TAILOR [9] are toolboxes for record linkage.”
Trio: A System for Integrated Management of Data, Accuracy, and Lineage (2004, background): “(In the closely related area of record linkage, e.g., [ EEV02 , JLM03], matching data items are identified but not merged.) In deduplication applications, some data items may include uncertainty (especially when merging data from multiple sources of varying quality), and the decision to merge two it”
Evaluation of entity resolution approaches on real-world match problems (2010, background): “, comparative evaluations of different string similarity metrics [11], [19], blocking approaches [3], [14], and clustering algorithms [17].”
A cost-based model and effective heuristic for repairing constraints by value modification (2005, background): “, [9, 20, 28]), also known as “duplicate removal” or “merge-purge”, and refers to the task of linking pairs of records that refer to the same entity in different data sets.”
A taxonomy of privacy-preserving record linkage techniques (2013, methodology): “Support vector machines and decision trees are two popular supervised learning techniques that have been employed for record linkage [55,61,62].”
A Survey of Indexing Techniques for Scalable Record Linkage and Deduplication (2012, methodology): “Many of them are based on machine learning approaches [10], [20], [28], [29], [30], [31].”
Large-Scale Deduplication with Constraints Using Dedupalog (2009, background): “Most of the existing work on deduplication focuses on record matching; record matching outputs candidate pairs of duplicate records by identifying pairs of similar records [42], [43], [44], [38], which may not be transitively closed.”
A Comparison of Fast Blocking Methods for Record Linkage (2003, methodology): “These results show trends similar to those produced by TAILOR [4] under a similar experimental framework.”
Frameworks for entity matching: A comparison (2010, methodology): “Hybrid frameworks TAILOR: TAILOR [24] is a toolbox for record linkage supporting numerical and rule-based combination approaches for multiple matchers without training as well as training-based.”
Data Matching (2012, methodology): “The ‘RecOrd LinkAge Toolbox’ (TAILOR) was developed at Purdue University in the early 2000 [102] as an extensible toolbox that allows different indexing, comparison, and classification techniques, as well as various evaluation methods, to be integrated.”
Duplicate Record Detection: A Survey (2007, methodology): “TAILOR [92] is a flexible record matching toolbox which allows the users to apply different duplicate detection methods on the data sets.”