Avoiding the Disk Bottleneck in the Data Domain Deduplication File System

By Benjamin Zhu, Kai Li, Hugo R. Patterson. USENIX Conference on File and Storage Technologies, 2008

888 citations recorded by the source corpus.

View Paper: https://www.semanticscholar.org/paper/0ce479229630e55e732597cf9b2aeb5018aae4c2

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Wen Xia
2. Feng Chen
3. Kiran Srinivasan
4. Hong Jiang
5. Biplob K. Debnath
6. Tian Luo
7. Dan Feng
8. Xiaodong Zhang

Citations in the corpus

• iDedup: latency-aware, inline data deduplication for primary storage (2012, methodology): “All prior deduplication work focuses on either: i) throughput sensitive archival and backup systems [8] [9] [15] [21] [26] [39] [41]; or ii) latency sensitive primary systems that deduplicate data offline during idle time [1] [11] [16]; or iii) file systems with inline deduplication, but agnostic…”
• A Performance Evaluation and Examination of Open-Source Erasure Coding Libraries for Storage (2009, background): “Storage companies such as Allmydata [1], Cleversafe [7], Data Domain [36], Network Appliance [22] and Panasas [32] are all delivering products that use erasure codes beyond RAID-5 for data availability.”
• Extreme Binning: Scalable, parallel deduplication for chunk-based file backup (2009, methodology): “Zhu et al.'s approach [9] always produces perfect deduplication but relies heavily on inherent data locality for its cache to be effective to improve throughput.”
• Sparse Indexing: Large Scale, Inline Deduplication Using Sampling and Locality (2009, methodology): “This problem has been addressed in the literature by Zhu et al. [28], who tackle it by using an in-memory Bloom Filter and caching index fragments, where each fragment indexes a set of chunks found together in the input.”
• CAFTL: A Content-Aware Flash Translation Layer Enhancing the Lifespan of Flash Memory based Solid State Drives (2011, methodology): “We have also considered using a Bloom filter [12] for pre-screening, like in the DataDomain file system [47], but found it inapplicable to CAFTL, because it requires multiple hashings and the summary vector cannot be updated when a fingerprint is removed.”
• ChunkStash: Speeding Up Inline Storage Deduplication Using Flash Memory (2010, methodology): “[41], prefetching of chunk index portions into RAM is shown to achieve RAM hit ratios of close to 99% on the evaluated datasets.”
• Backyard Cuckoo Hashing: Constant Worst-Case Operations with a Succinct Representation (2009, methodology): “Our approach is based on hashing elements using permutations instead of functions.”
• A Comprehensive Study of the Past, Present, and Future of Data Deduplication (2016, methodology): “Upon the lookup of fingerprint “3b” of file C, DDFS will prefetch the fingerprints {3b, a7, 2f, 5c} and preserve this locality in the RAM, which helps reduce the Vol. 104, No. 9, September 2016 | Proceedings of the IEEE 1691 accesses to the on-disk index when looking up the fingerprints of “a7” and ”
• Message-Locked Encryption for Lock-Dependent Messages (2013, unknown): “example, [28,35,26]).”