A protocol-independent technique for eliminating redundant network traffic

By Neil T. Spring, David Wetherall. Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication, 2000

395 citations recorded by the source corpus.

We present a technique for identifying repetitive information transfers and use it to analyze the redundancy of network traffic. Our insight is that dynamic content, streaming media and other traffic that is not caught by today's Web caches is nonetheless likely to derive from similar information. We have therefore adapted similarity detection techniques to the problem of designing a system to eliminate redundant transfers. We identify repeated byte ranges between packets to avoid retransmitting the redundant data. We find a high level of redundancy and are able to detect repetition that Web proxy caches are not. In our traces, after Web proxy caching has been applied, an additional 39% of the original volume of Web traffic is found to be redundant. Moreover, because our technique makes no assumptions about HTTP protocol syntax or caching semantics, it provides immediate benefits for other types of content, such as streaming media, FTP traffic, news and mail.

Subfields: Artificial Intelligence, Computer Networks and Communications

View Paper: https://www.semanticscholar.org/paper/39b807ec8d296ea60992ed462a1742da630ed1cb

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

Citations in the corpus

The PARSEC benchmark suite: Characterization and architectural implications (2008, methodology): “The reason for the inclusion of this kernel is that deduplication has become a mainstream method to compress storage footprints for new-generation backup storage systems[36] and to compress communication data for newgeneration bandwidth optimized networking appliances[39].”
A Comprehensive Study of the Past, Present, and Future of Data Deduplication (2016, neutral): “[183], SmartRE [184], EndRE [81] SSD, multimedia, etc.”
Benchmarking modern multiprocessors (2011, methodology): “The reason for the inclusion of this kernel is that deduplication has become a mainstream method to reduce storage footprints for new-generation backup storage systems [76] and to compress communication data for new-generation bandwidth optimized networking appliances [85].”