MacroBase: Prioritizing Attention in Fast Data
By Peter Bailis, Edward Gan, Samuel Madden, D. Narayanan, Kexin Rong, Sahaana Suri. 2016
140 citations recorded by the source corpus.
As data volumes continue to rise, manual inspection is becoming increasingly untenable. In response, we present MacroBase, a data analytics engine that prioritizes end-user attention in high-volume fast data streams. MacroBase enables efficient, accurate, and modular analyses that highlight and aggregate important and unusual behavior, acting as a search engine for fast data. MacroBase is able to deliver order-of-magnitude speedups over alternatives by optimizing the combination of explanation and classification tasks and by leveraging a new reservoir sampler and heavy-hitters sketch specialized for fast data streams. As a result, MacroBase delivers accurate results at speeds of up to 2M events per second per query on a single core. The system has delivered meaningful results in production, including at a telematics company monitoring hundreds of thousands of vehicles.
Subfields: Artificial Intelligence, Signal Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Sanjay Krishnan
- Eugene Wu
- Radwa El Shawi
- Tom Diethe
- Sherif Sakr
- Mohamed Maher
- Kevin Hsieh
- Neil D. Lawrence
In Signal Processing
- Eugene Wu
- Neil D. Lawrence
- Onur Mutlu
- Borja Balle
- Sherif Sakr
- Radwa El Shawi
- Phillip B. Gibbons
- Mohamed Maher
Citations in the corpus
- Automated Machine Learning: State-of-The-Art and Open Challenges (2019, background): “MacroBase[3] is an open source framework for data analytics, and search engine for big, and fast data streams.”
- Continual Learning in Practice (2019, background): “MacroBase [4] is an analytic monitoring engine designed to prioritise human attention in large-scale datasets and data streams.”
- PALM: Machine Learning Explanations For Iterative Debugging (2017, methodology): “We believe that coupling explanations with anomaly detection may be a viable next step, as in the MacroBase project [3].”
- Focus: Querying Large Video Datasets with Low Latency and Low Cost (2018, background): “, [1,18,22,25,31,32,61,66,86, 87, 95]) and specific to video stream analytics (e.”