Controlling False Discoveries During Interactive Data Exploration

By Zheguang Zhao, Lorenzo De Stefani, Emanuel Zgraggen, Carsten Binnig, Eli Upfal, Tim Kraska. 2016

75 citations recorded by the source corpus.

Recent tools for interactive data exploration significantly increase the chance that users make false discoveries. They allow users to (visually) examine many hypotheses and make inference with simple interactions, and thus incur the issue commonly known in statistics as the "multiple hypothesis testing error." In this work, we propose a solution to integrate the control of multiple hypothesis testing into interactive data exploration systems. A key insight is that existing methods for controlling the false discovery rate (such as FDR) are not directly applicable to interactive data exploration. We therefore discuss a set of new control procedures that are better suited for this task and integrate them in our system, QUDE. Via extensive experiments on both real-world and synthetic data sets we demonstrate how QUDE can help experts and novice users alike to efficiently control false discoveries.

Subfields: Computer Vision and Pattern Recognition, Economics and Econometrics, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Yuji Roh
2. Steven Euijong Whang
3. Sudip Roy
4. Neoklis Polyzotis
5. Sherif Sakr
6. Radwa El Shawi
7. Mohamed Maher
8. Jae-Gil Lee

In Economics and Econometrics

1. Martin A. Zinkevich

In Signal Processing

1. Yuji Roh
2. Steven Euijong Whang
3. Sherif Sakr
4. Radwa El Shawi
5. Mohamed Maher

Citations in the corpus

• Automated Machine Learning: State-of-The-Art and Open Challenges (2019, methodology): “QUDE [133] presents a solution using a novel α -investing technique to the multiple hypothesis testing error introduced by previous tools which is making several false discoveries based on inference seen by users from simple interactions visualized by these tools.”
• Data Lifecycle Challenges in Production Machine Learning (2018, methodology): “The QUDE system [14, 75] takes a statistical approach and provides an interactive data exploration framework that uses multiple hypothesis testing to control the false positives.”
• Data collection and quality challenges in deep learning: a data-centric AI perspective (2021, background): “CUDE[184] controls false discovery in the context of multiple hypothesis testing for visual interactive data exploration.”