Data Validation for Machine Learning

By Eric Breck, Neoklis Polyzotis, Sudip Roy, Steven Euijong Whang, Martin A. Zinkevich. USENIX workshop on Tackling computer systems problems with machine learning techniques, 2019

369 citations recorded by the source corpus.

View Paper: https://www.semanticscholar.org/paper/2d57b38ff6fa651cc932a29f0c7ca72e9f7a8cf9

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Bridging the Gap Between Ethics and Practice (2020, background): “The history of testing should be recorded to enable reconstruction and document how and by whom problems were repaired [11, 126].”
“Everyone wants to do the model work, not the data work”: Data Cascades in High-Stakes AI (2021, methodology): “Researchers have created several tools to support the creation of ML ‘pipelines’ and make these workfows manageable [21, 54, 70, 72, 76].”
Data collection and quality challenges in deep learning: a data-centric AI perspective (2021, methodology): “Tensorﬂow Data Validation (TFDV) [36, 59] assumes a continuous training setting where input data periodically streams in as shown in Figure 3.”
The Eﬀects of Data Quality on Machine Learning Performance (2022, methodology): “For instance, the validation system implemented by Breck et al. [9] and the similar system by Schelter et al. [62] focus on validating serving data given a classiﬁcation pipeline as a black box.”
Machine Learning Testing: Survey, Landscapes and Horizons (2019, methodology): “[45] used synthetic training data that adhere to schema constraints to trigger the hidden assumptions in the code that do not agree with the constraints.”
Lakehouse: A New Generation of Open Platforms that Unify Data Warehousing and Advanced Analytics (2021, background): “In addition, ML and data science applications suffer from the same data management problems that classical applications do, such as data quality, consistency, and isolation [17, 27, 31], so there is immense value in bringing DBMS features to their data.”
Challenges in Deploying Machine Learning: A Survey of Case Studies (2020, background): “ard for training agents. In addition, the dataset itself also needs to be constantly validated to ensure data errors do not creep into the pipeline and do not affect the overall quality. Breck et al. [49] argue that one of the most common scenarios when issues in data can go unnoticed is the setup w”
Software Engineering for AI-Based Systems: A Survey (2021, background): “These frameworks normally focus on specific quality aspects of ML products, such as allowability, achievability, robustness, avoidability and improvability [147], safety [46, 135], specific safety issues like forward collision mitigation based on the ISO 22839 standard [55], security [50], algorithm”
Overview and Importance of Data Quality for Machine Learning Tasks (2020, methodology): “We further plan to discuss few automated data cleaning approaches proposed in the literature [1] and finally end the topic by illustrating a real life use-case of deployed data cleaning system in Google [2] and its benefits.”
The effects of data quality on machine learning performance on tabular data (2022, unknown): “For instance, the validation system implemented by Breck et al. [10] and the similar system by Schelter et al. [63] focus on validating serving data given a classification pipeline as a black box.”