Sparse Indexing: Large Scale, Inline Deduplication Using Sampling and Locality

472 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

A Comprehensive Study of the Past, Present, and Future of Data Deduplication (2016, methodology): “For example, some data deduplication systems [11], [13], [58] report that the accessing throughput to the on-disk fingerprint-index is about 1–6 MB/s, which becomes a severe performance bottleneck in these systems.”
CAFTL: A Content-Aware Flash Translation Layer Enhancing the Lifespan of Flash Memory based Solid State Drives (2011, unknown): “[11,24,30,45,47],which is designed for backup/archival systems.”
Extreme Binning: Scalable, parallel deduplication for chunk-based file backup (2009, background): “Lillibridge et al. [10] batch up chunks into large segments, on the order of 10 MB.”
ChunkStash: Speeding Up Inline Storage Deduplication Using Flash Memory (2010, methodology): “Similar to [41] and unlike [30], our system targets complete deduplication and ensures that no duplicate chunks exist in the system after deduplication.”
iDedup: latency-aware, inline data deduplication for primary storage (2012, methodology): “…EMC DDFS [41], throughput EMC Cluster [8] sensitive DeepStore [40], (No motivation NEC HydraStor [9], for systems in Venti [31], SiLo [39], this category) Sparse Indexing [21], ChunkStash [7], Foundation [32], Symantec [15], EMC Centera [24], GreenBytes [13] Table 1: Table of related work:.”