Large-Scale Collective Entity Matching

By Vibhor Rastogi, Nilesh N. Dalvi, Minos N. Garofalakis. 2011

118 citations recorded by the source corpus.

There have been several recent advancements in Machine Learning community on the Entity Matching (EM) problem. However, their lack of scalability has prevented them from being applied in practical settings on large real-life datasets. Towards this end, we propose a principled framework to scale any generic EM algorithm. Our technique consists of running multiple instances of the EM algorithm on small neighborhoods of the data and passing messages across neighborhoods to construct a global solution. We prove formal properties of our framework and experimentally demonstrate the effectiveness of our approach in scaling EM algorithms.

Subfields: Artificial Intelligence, Management Science and Operations Research, Information Retrieval & Search, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

In Information Retrieval & Search

In Natural Language Processing

Citations in the corpus

SIGMa: simple greedy matching for aligning large knowledge bases (2012, methodology): “Some scalable solutions for collective entity resolution were proposed recently [1, 25], though they did not implement a 1–1 matching constraint, and their implementation can be a complex software engineering endeavor in contrast to the simplicity of our approach.”
Fast Learning from Distributed Datasets without Entity Matching (2016, background): “The various metrics of ER encompass lots of different parameters, including generality, accuracy, soundness, scalability, parallelizability [Rastogi et al., 2011].”
Joint Entity Resolution (2012, background): “Although several recent works [17], [18] have proposed general scalable ER algorithms for a single entity type of data, they do not discuss”
LINDA: distributed web-of-data-scale entity matching (2012, background): “A recent proposal for scaling up joint inference partitions the input into neighborhoods, and then parallel computations per neighborhood are periodically reconciled by message passing [10].”
Data Matching (2012, background): “Recent work has aimed to improve the scalability of collective classification techniques by running a collective matching process many times on small subsets of records that are in the same neighbourhood of the data [225].”