Predicting the Computational Cost of Deep Learning Models

By Daniel Justus, John Brennan, Stephen Bonner, A. Mcgough. 2018

287 citations recorded by the source corpus.

Deep learning is rapidly becoming a go-to tool for many artificial intelligence problems due to its ability to outperform other approaches and even humans at many problems. Despite its popularity we are still unable to accurately predict the time it will take to train a deep learning network to solve a given problem. This training time can be seen as the product of the training time per epoch and the number of epochs which need to be performed to reach the desired level of accuracy. Some work has been carried out to predict the training time for an epoch – most have been based around the assumption that the training time is linearly related to the number of floating point operations required. However, this relationship is not true and becomes exacerbated in cases where other activities start to dominate the execution time. Such as the time to load data from memory or loss of performance due to non-optimal parallel execution. In this work we propose an alternative approach in which we train a deep learning network to predict the execution time for parts of a deep learning network. Timings for these individual parts can then be combined to provide a prediction for the whole execution time. This has advantages over linear approaches as it can model more complex scenarios. But, also, it has the ability to predict execution times for scenarios unseen in the training data. Therefore, our approach can be used not only to infer the execution time for a batch, or entire epoch, but it can also support making a well-informed choice for the appropriate hardware and model.

Subfields: Artificial Intelligence, Hardware and Architecture, Management Science and Operations Research

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Hardware and Architecture

In Management Science and Operations Research

Citations in the corpus

CODA-Prompt: COntinual Decomposed Attention-Based Prompting for Rehearsal-Free Continual Learning (2022, background): “This could incur high financial [26] and environmental [34] costs if the replacement is frequent.”
Best Practices for Scientific Research on Neural Architecture Search (2019, background): “, 2019), which will be hardware-dependent, or hardware-independent proxies, such as number of floating point operations (FLOPS), which is not always linearly correlated with the runtime (Justus et al., 2018), or in epochs (if these are roughly similarly expensive for different architectures and hype”
Regular Polytope Networks (2021, background): “The time it takes to train a neural network model to address a classification problem is typically considered as the product of the training time per epoch and the number of epochs that need to be performed to reach the desired level of accuracy [54].”
A Closer Look at Rehearsal-Free Continual Learning * (2022, background): “This may lead to high financial [24] and environmental [32] costs after numerous replacements.”
A survey of the recent architectures of deep convolutional neural networks (2019, background): “However, the training of deep and high capacity architectures is still a significant overhead on memory usage and computational resources (Lacey et al. 2016; Sze et al. 2017; Justus et al. 2019).”