Big data integration

By Xin Luna Dong, Divesh Srivastava. 2013

588 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In ML Systems & Hardware

Citations in the corpus

Snorkel: Rapid Training Data Creation with Weak Supervision (2017, background): “Related Data Management Problems: Researchers have considered related problems in data management, such as data fusion [15,40] and truth discovery [28].”
Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI (2019, neutral): “As presented in [423] , the Big Data era presents an environment in which these premises cannot be taken for granted, and methods to board Big Data fusion (as that illustrated in Fig.”
Big data analytics capabilities: a systematic literature review and research agenda (2017, methodology): “Hence, big data used for business decisions should be authenticated and have passed through strict quality-compliance procedures before being analyzed (Dong and Srivastava 2013; Gandomi and Haider 2015).”
A sample-and-clean framework for fast and accurate query processing on dirty data (2014, background): “Real-world data is commonly integrated from multiple sources, and the integration process may lead to a variety of data errors, such as incorrect values and duplicate representations of the same real-world entity [21].”
Review of Bridge Structural Health Monitoring Aided by Big Data and Artificial Intelligence: From Condition Assessment to Damage Detection (2020, unknown): “Data integration that merges heterogeneous data from different sources was a step of data preprocessing (Han et al. 2011), then became somewhat different after big data was introduced (Dong and Srivastava 2013).”
Data Lake Management: Challenges and Opportunities (2019, unknown): “We will survey some Big Data Integration techniques that tackle dynamic data which may be of very poor data quality [10] to consider how they apply to data lakes.”
Snuba: Automating Weak Supervision to Label Training Data (2018, unknown): “Areas like data fusion [13, 41, 38] and truth discovery [30] also look at the problem of estimating how reliable different data sources are while utilizing probabilistic graphical models like Snuba.”
Big Data and cloud computing: innovation opportunities and challenges (2017, background): “Dong and Divesh (2015) summarized the data integration challenges of schema mapping, record linkage and data fusion.”
SampleClean: Fast and Reliable Analytics on Dirty Data (2015, background): “Real-world data are commonly integrated from multiple sources, and the integration process may lead to a variety of errors [20].”
Snorkel DryBell: A Case Study in Deploying Weak Supervision at Industrial Scale (2018, background): “Related problems in data systems include data fusion [11, 33] and truth discovery [19].”
Resolving conflicts in heterogeneous data by truth discovery and source reliability estimation (2014, methodology): “Existing truth discovery approaches [3, 8, 9, 11, 12, 14–16, 18, 21–24] and the proposed method try to estimate source reliability in an unsupervised manner.”
Snorkel: rapid training data creation with weak supervision (2019, background): “Related Data Management Problems Researchers have considered related problems in data management, such as data fusion [15,48] and truth discovery [32].”
Detecting Data Errors: Where are we and what needs to be done? (2016, background): “In the last two decades there has been intensive research in developing data cleaning algorithms and tools [10,11,14, • Rule-based detection algorithms [1,6,15,16,25,33] that can be embedded into frameworks, such as Nadeef [8, 23], where a rule can vary from a simple “not null” constraint to…”
Are Key-Foreign Key Joins Safe to Avoid when Learning High-Capacity Classifiers? (2017, background): “, integrating features from different data types in recommendation systems [21], sensor fusion [22], dimensionality reduction during feature fusion [14], and controlling data quality during data fusion [11].”
Toward a System Building Agenda for Data Integration (and Data Science) (2018, background): “Over the past few decades, DI has received much attention (e.g., [37, 29, 31, 20, 34, 33, 6, 17, 39, 22, 23, 5, 8, 36, 15, 35, 4, 25, 38, 26, 32, 19, 2, 12, 11, 16, 2, 3]).”
Understandable Big Data: A survey (2015, methodology): “Our work is similar to [9] in the approach.”
SLiMFast: Guaranteed Results for Data Fusion and Source Reliability (2015, background): “Thus, data fusion—the task of resolving conflicts across sources by estimating their trustworthiness—has emerged as a key element of many data integration pipelines [14].”
A Survey on Deep Learning in Big Data (2017, background): “Many open research topics are associated with data integration like real-time data access, the performance of the system, and overlapping of the same data [44].”