Load Shedding in a Data Stream Manager

By Nesime Tatbul, Ugur Çetintemel, S. Zdonik, Mitch Cherniack, M. Stonebraker. Very Large Data Bases Conference, 2003

680 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Networks and Communications

View Paper: https://www.semanticscholar.org/paper/4783c303f45d78323e1206c962ce6fcea57e724d

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

Citations in the corpus

The CQL continuous query language: semantic foundations and query execution (2006, background): “System operators isolate the data operators from lower-level issues such as asynchronous and out-oforder arrival of streams, load shedding [9, 40], and external stream data formats.”
The Design of the Borealis Stream Processing Engine (2005, background): “Several groups have developed working prototypes [1, 4, 16] and many papers have been published on detailed aspects of the technology such as data models [2, 5, 46], scheduling [8, 15], and load shedding [9, 20, 44].”
Issues in data stream management (2003, background): “memory usage: Sampling and load shedding [62] may be used to decrease memory usage by increasing the error.”
Data Streams: Models and Algorithms (Advances in Database Systems) (2006, background): “[21] Henzinger M., Raghavan P. and Rajagopalan S. (1998), Computing on data streams , Technical Note 1998-01 1, Digital Systems Research Center, Palo Alto, CA.”
Streaming Pattern Discovery in Multiple Time-Series (2005, background): “The new challenges have made researchers re-think many parts of traditional DBMS design in the streaming context, especially on query processing using correlated attributes [14], scheduling [6, 8], load shedding [35, 12] and memory requirements [5].”
Live Video Analytics at Scale with Approximation and Delay-Tolerance (2017, background): “Aurora, Medusa, and Borealis [19, 33, 37] and follow-up works [78, 79, 81, 82, 88] use QoS graphs to capture lag and sampling rate but they consider them separately and do not trade-off between them, a key aspect in our solution.”
Fault-tolerance in the Borealis distributed stream processing system (2005, background): “Much work has been done on data models and operators [1, 6, 16, 28, 41], efficient processing [7, 8, 12, 30], and resource management [13, 17, 30, 35, 38] for SPEs.”
Stale View Cleaning: Getting Fresh Answers from Stale Materialized Views (2015, background): “Sampling has been explored in the streaming community, and a similar idea of sampling from incoming updates has also been applied in stream processing [16,44,47].”
Operator Scheduling in a Data Stream Manager (2003, background): “Second, Aurora employs a load shedding mechanism (not described in this paper but can be found in [8, 21]) that is initiated when an overload situation is detected and that selectively sheds load to get rid of excess load in a way that least degrades the QoS.”
Load shedding for aggregation queries over data streams (2004, methodology): “The approach taken in [22] is different than ours in that it relies on quality of service functions that specify the utility of processing each tuple from a data stream.”