Feasibility of a serverless distributed file system deployed on an existing set of desktop PCs

By William J. Bolosky, John R. Douceur, David Ely, Marvin Theimer. Measurement and Modeling of Computer Systems, 2000

582 citations recorded by the source corpus.

We consider an architecture for a serverless distributed file system that does not assume mutual trust among the client computers. The system provides security, availability, and reliability by distributing multiple encrypted replicas of each file among the client machines. To assess the feasibility of deploying this system on an existing desktop infrastructure, we measure and analyze a large set of client machines in a commercial environment. In particular, we measure and report results on disk usage and content; file activity; and machine uptimes, lifetimes, and loads. We conclude that the measured desktop infrastructure would passably support our proposed system, providing availability on the order of one unfilled file request per user per thousand days.

Subfields: Computer Networks and Communications

View Paper: https://www.semanticscholar.org/paper/d1da7bbfb70b0fb6927147e32e0e71fd60ae8ba1

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

Citations in the corpus

Delay aware querying with Seaweed (2006, unknown): “Simulated endsystem availability is based on the Farsite trace of endsystem availability gathered over approximately 4 weeks in July/August 1999 in the Microsoft corporate network [8].”
MPICH-V: Toward a Scalable Fault Tolerant MPI for Volatile Nodes (2002, background): “A recent study of the availability of desktop machines within a large industry network ( 64,000 machines) [5], which is a typical Large Scale Virtual PC Farms targeted for Global Computing platforms in industry and university, demonstrates that from 5% to 10% of the machines become unreachable in a ”
Pastry: Scalable, distributed object location and routing for large-scale peer-to- (2001, background): “There are a number of research projects focusing on peer-topeer storage utilities, such as FarSite [4] and Oceanstore [5].”
Reclaiming space from duplicate files in a serverless distributed file system (2002, methodology): “Background – the Farsite file system Farsite [8] is a scalable, serverless, distributed file system under development at Microsoft Research.”
Network Coding for Distributed Storage Systems (2007, methodology): “A number of recent studies [4], [18], [7], [21], [2], [27] have designed and evaluated large-scale, peer-to-peer distributed storage systems.”
Pond: The OceanStore Prototype (2003, unknown): “FarSite [3] aims to build an enterprise-scale distributed file system, using Byzantine fault-tolerance for directories only.”
The Sybil Attack (2002, background): “Many systems [3, 4, 8, 10, 17, 18, 29, 34, 36] replicate computational or storage tasks among several remote sites to protect against integrity violations (data loss).”
A Survey of Peer-to-Peer Security Issues (2002, background): “In the Farsite study [16], the authors noted that hard drives are often relatively empty.”
Glacier: highly durable, decentralized storage despite massive correlated failures (2005, background): “[8] report an expected machine lifetime of 290 days and low short-term churn, which would reduce the maintenance overhead considerably.”
A survey and comparison of peer-to-peer overlay network schemes (2005, methodology): “CAN could be used in large scale storage management systems such as the OceanStore [17], Farsite [18], and Publius [19].”
Total Recall: System Support for Automated Availability Management (2004, unknown): “The File Sharing trace is a scaled down version of a trace of host availability in the Overnet file sharing system [4], while the File System availability trace is synthetically generated using the host availability distribution in [6].”
A scalable content-addressable network (2001, unknown): “These systems all require efﬁcient insertion and retrieval of content in a large distributed storage infrastructure; a scalable indexing mechanism is an essential component of such an infrastructure.”
High Availability in DHTs: Erasure Coding vs. Replication (2005, background): “[6] on their study of the availability of 51,663 desktop PCs at Microsoft Corporation over the course of 35 days by pinging a fixed set of machines every hour.”
A Comprehensive Study of the Past, Present, and Future of Data Deduplication (2016, background): “It is worth noting that file-level deduplication was proposed earlier [10], [56], but that technique was subsequently overshadowed by chunk-level deduplication due to the latter’s better compression performance [5], [12], [13], [57].”
MR-PDP: Multiple-Replica Provable Data Possession (2008, methodology): “These goals include availability [ 9 ], preservation [20], and censorship resistance [28].”
In VINI veritas: realistic and controlled network experimentation (2006, methodology): “) on a fixed physical infrastructure? The answer to this question and other insights we glean from the design and implementation of VINI will provide important lessons for the design of experimental infrastructures such as the National Science Foundation’s Global Environment for Network Innovations ”
This paper is included in the Proceedings of the 2022 USENIX Annual Technical Conference. (unknown): “…“ if the proposer receives a response from a majority of acceptors... ” In practice, many projects started with the asynchronous callback style, but the growing size of codebase and the accompanying cognitive load of callbacks made the developers change to a synchronous code style [15,18].”
Eclipse Attacks on Overlay Networks: Threats and Defenses (2006, background): “[3] for a corporate environment, where the average session time was reported as 37.”