Management of an academic HPC cluster: The UL experience
By Sébastien Varrette, Pascal Bouvry, Hyacinthe Cartiaux, F. Georgatos. International Symposium on High Performance Computing Systems and Applications, 2014
311 citations recorded by the source corpus.
The intensive growth of processing power, data storage and transmission capabilities has revolutionized many aspects of science. These resources are essential to achieve high-quality results in many application areas. In this context, the University of Luxembourg (UL) operates since 2007 an High Performance Computing (HPC) facility and the related storage by a very small team. The aspect of bridging computing and storage is a requirement of UL service - the reasons are both legal (certain data may not move) and performance related. Nowadays, people from the three faculties and/or the two Interdisciplinary centers within the UL, are users of this facility. More specifically, key research priorities such as Systems Bio-medicine (by LCSB) and Security, Reliability & Trust (by SnT) require access to such HPC facilities in order to function in an adequate environment. The management of HPC solutions is a complex enterprise and a constant area for discussion and improvement. The UL HPC facility and the derived deployed services is a complex computing system to manage by its scale: at the moment of writing, it consists of 150 servers, 368 nodes (3880 computing cores) and 1996 TB of shared storage which are all configured, monitored and operated by only three persons using advanced IT automation solutions based on Puppet [1], FAI [2] and Capistrano [3]. This paper covers all the aspects in relation to the management of such a complex infrastructure, whether technical or administrative. Most design choices or implemented approaches have been motivated by several years of experience in addressing research needs, mainly in the HPC area but also in complementary services (typically Web-based). In this context, we tried to answer in a flexible and convenient way many technological issues. This experience report may be of interest for other research centers and universities belonging either to the public or the private sector looking for good if not best practices in cluster architecture and management.
Subfields: Computer Networks and Communications, Information Systems, Information Systems and Management
View Paper: https://www.semanticscholar.org/paper/333f6aa940113abc9ac7f779bef6bcef7eee60e2
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computer Networks and Communications
In Information Systems
In Information Systems and Management
Citations in the corpus
- Osiris: Hunting for Integer Bugs in Ethereum Smart Contracts (2018, methodology): “The experiments presented in this paper were carried out using the HPC facilities of the University of Luxembourg [34] – see https://hpc.”
- Generating Multi-Categorical Samples with Generative Adversarial Networks (2018, methodology): “The experiments presented in this paper were carried out using the HPC facilities of the University of Luxembourg (Varrette et al., 2014) – see https://hpc.”