HyperDrive: exploring hyperparameters with POP scheduling

41 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Population Based Training of Neural Networks (2017, methodology): “Many of these Bayesian optimisation approaches can be further sped up by attempting to parallelise them – training independent models in parallel to quicker update the Bayesian model (though potentially introducing bias) (Shah & Ghahramani, 2015; González et al., 2016; Wu & Frazier, 2016; Rasley et ”
Zico: Efficient GPU Memory Sharing for Concurrent DNN Training (2021, methodology): “As hyperparameters constitute a large search space, there are several popular tools such as Hyperdrive [35] and HyperOpt [4] that automate hyperparameter optimization and construct a new model with the best (or desired) quality for users.”
Themis: Fair and Efficient GPU Cluster Scheduling for Machine Learning Workloads (2019, methodology): “We modify the Submarine client to support submitting a group of ML training jobs as required by hyperparameter exploration ML apps [18, 21].”
Gandiva: Introspective Cluster Scheduling for Deep Learning (2018, background): “Google Vizier [21], HyperDrive [41], and TuPAQ [45] focus more on the system design of AutoML.”
Salus: Fine-Grained GPU Sharing Primitives for Deep Learning Applications (2019, background): “, AutoML [18,36, 42]) further emphasizes the need to improve GPU utilization.”
Learning Intrinsic Sparse Structures within Long Short-term Memory (2017, methodology): “We used HyperDrive by Rasley et al. (2017) to explore the hyperparameter of λ .”