Tachyon: Reliable, Memory Speed Storage for Cluster Computing Frameworks

By Haoyuan Li, A. Ghodsi, Matei A. Zaharia, Scott Shenker, Ion Stoica. ACM Symposium on Cloud Computing, 2014

371 citations recorded by the source corpus.

Tachyon is a distributed file system enabling reliable data sharing at memory speed across cluster computing frameworks. While caching today improves read workloads, writes are either network or disk bound, as replication is used for fault-tolerance. Tachyon eliminates this bottleneck by pushing lineage, a well-known technique, into the storage layer. The key challenge in making a long-running lineage-based storage system is timely data recovery in case of failures. Tachyon addresses this issue by introducing a checkpointing algorithm that guarantees bounded recovery cost and resource allocation strategies for recomputation under commonly used resource schedulers. Our evaluation shows that Tachyon outperforms in-memory HDFS by 110x for writes. It also improves the end-to-end latency of a realistic workflow by 4x. Tachyon is open source and is deployed at multiple companies.

Subfields: Computer Networks and Communications, Information Systems

View Paper: https://www.semanticscholar.org/paper/d48e362d8c25179cf05f737d64d070c64c57186b

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Systems

  1. Joshua Zhexue Huang
  2. Ruslan Dautov
  3. Salman Salloum
  4. Patrick Xiaogang Peng
  5. Xiaojun Chen

Citations in the corpus