Understanding failures in petascale computers

By Bianca Schroeder, Garth A. Gibson. 2007

489 citations recorded by the source corpus.

With petascale computers only a year or two away there is a pressing need to anticipate and compensate for a probable increase in failure and application interruption rates. Researchers, designers and integrators have available to them far too little detailed information on the failures and interruptions that even smaller terascale computers experience. The information that is available suggests that application interruptions will become far more common in the coming decade, and the largest applications may surrender large fractions of the computer's resources to taking checkpoints and restarting from a checkpoint after an interruption. This paper reviews sources of failure information for compute clusters and storage systems, projects failure rates and the corresponding decrease in application effectiveness, and discusses coping strategies such as application-level checkpoint compression and system level process-pairs fault-tolerance for supercomputing. The need for a public repository for detailed failure and interruption records is particularly concerning, as projections from one architectural family of machines to another are widely disputed. To this end, this paper introduces the Computer Failure Data Repository and issues a call for failure history data to publish in it.

Subfields: Computer Networks and Communications

View Paper: https://www.semanticscholar.org/paper/78df7a454cbc13c919323cc25764653553b959fb

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

Citations in the corpus

Toward Exascale Resilience (2009, background): “Several analyses have been made concerning the root cause behind failures (Lu, 2005; Oliner and Stearley, 2007; Schroeder and Gibson, 2007).”
PLFS: a checkpoint filesystem for parallel applications (2009, background): “Even using optimal checkpoint frequencies [9], checkpointing has become the driving workload for parallel file systems and the challenge it imposes grows with each successively larger supercomputer [32, 42].”
Evaluating the viability of process replication reliability for exascale systems (2011, background): “For example, several independent studies have concluded that potential exascale systems could spend more than 50% of their time reading and writing checkpoints [12, 28, 38].”
Six degrees of scientific data: reading patterns for extreme scale science IO (2011, background): “First, as HPC applications run with higher process counts, we see increased frequencies and sizes in the checkpoints needed for fault tolerance [23].”
FTI: High performance Fault Tolerance Interface for hybrid systems (2011, background): “While HPC systems are built with highly reliable components, such large machines are expected to experience failures on a regular basis [9, 33, 38].”