Why Should We Add Early Exits to Neural Networks?

By Simone Scardapane, Michele Scarpiniti, Enzo Baccarelli, Aurelio Uncini. 2020

152 citations recorded by the source corpus.

Deep neural networks are generally designed as a stack of differentiable layers, in which a prediction is obtained only after running the full stack. Recently, some contributions have proposed techniques to endow the networks with early exits, allowing to obtain predictions at intermediate points of the stack. These multi-output networks have a number of advantages, including (i) significant reductions of the inference time, (ii) reduced tendency to overfitting and vanishing gradients, and (iii) capability of being distributed over multi-tier computation platforms. In addition, they connect to the wider themes of biological plausibility and layered cognitive reasoning. In this paper, we provide a comprehensive introduction to this family of neural networks, by describing in a unified fashion the way these architectures can be designed, trained, and actually deployed in time-constrained scenarios. We also describe in-depth their application scenarios in 5G and Fog computing environments, as long as some of the open research questions connected to them.

Subfields: Artificial Intelligence, Computer Networks and Communications, Electrical and Electronic Engineering, Model Architectures, Efficiency & Compression

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

Yossi Matias

In Model Architectures

In Efficiency & Compression

Citations in the corpus

Fast Inference from Transformers via Speculative Decoding (2022, methodology): “These adaptive computation methods (e.g. Han et al., 2021; Sukhbaatar et al., 2019; Schuster et al., 2021; Scardapane et al., 2020; Bapna et al., 2020; Elbayad et al., 2019; Schwartz et al., 2020) aim to use less compute resources for easier inference steps.”
Surprisal-Triggered Conditional Computation with Neural Networks (2020, background): “In early exiting, a classiﬁer is attached to an intermediate layer of a network and trained to determine whether stopping at that layer will result in a misclassiﬁcation; if a misclassiﬁcation is not predicted, subsequent layers are not computed [13, 14, 15, 16, 17].”