FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search

By Bichen Wu, Xiaoliang Dai, Peizhao Zhang, Yanghan Wang, Fei Sun, Yiming Wu, Yuandong Tian, Péter Vajda, Yangqing Jia, Kurt Keutzer. 2018

1,467 citations recorded by the source corpus.

Designing accurate and efficient ConvNets for mobile devices is challenging because the design space is combinatorially large. Due to this, previous neural architecture search (NAS) methods are computationally expensive. ConvNet architecture optimality depends on factors such as input resolution and target devices. However, existing approaches are too resource demanding for case-by-case redesigns. Also, previous work focuses primarily on reducing FLOPs, but FLOP count does not always reflect actual latency. To address these, we propose a differentiable neural architecture search (DNAS) framework that uses gradient-based methods to optimize ConvNet architectures, avoiding enumerating and training individual architectures separately as in previous methods. FBNets (Facebook-Berkeley-Nets), a family of models discovered by DNAS surpass state-of-the-art models both designed manually and generated automatically. FBNet-B achieves 74.1% top-1 accuracy on ImageNet with 295M FLOPs and 23.1 ms latency on a Samsung S8 phone, 2.4x smaller and 1.5x faster than MobileNetV2-1.3 with similar accuracy. Despite higher accuracy and lower latency than MnasNet, we estimate FBNet-B's search cost is 420x smaller than MnasNet's, at only 216 GPU-hours. Searched for different resolutions and channel sizes, FBNets achieve 1.5% to 6.4% higher accuracy than MobileNetV2. The smallest FBNet achieves 50.2% accuracy and 2.9 ms latency (345 frames per second) on a Samsung S8. Over a Samsung-optimized FBNet, the iPhone-X-optimized model achieves a 1.4x speedup on an iPhone X. FBNet models are open-sourced at https://github. com/facebookresearch/mobile-vision.

Subfields: Computer Vision and Pattern Recognition, Electrical and Electronic Engineering, Efficiency & Compression

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Electrical and Electronic Engineering

In Efficiency & Compression

Citations in the corpus

Visual Transformers: Token-based Image Representation and Processing for Computer Vision (2020, background): “Recently, people use neural architecture search [42, 10, 38, 9, 36, 35] to optimize network’s performance within a search space that consists of existing convolution operators.”
The Efficiency Misnomer (2021, methodology): “However, counting training cost in terms of steps can be problematic as step count in itself is not a meaningful metric, and can be stretched arbitrarily with optimizers introducing multi-step lookahead schemes (Zhang et al., 2019).”
AutoML: A Survey of the State-of-the-Art (2019, methodology): “Another DNAS method, called FBNet [143], uses a latency lookup-table model to estimate the overall latency of a network based on the runtime of each operator and differentiate the latency of candidate operations.”
Group Knowledge Transfer: Collaborative Training of Large CNNs on the Edge (2020, neutral): “Our work also relates to efficient deep learning on edge devices, such as model compression [43, 44, 45], manually designed architectures (MobileNets [46], ShuffeNets [47], SqueezeNets [48]), or even efficient neural architecture search (EfficientNets [4], FBNet [49]).”
AutoGAN-Distiller: Searching to Compress Generative Adversarial Networks (2020, neutral): “We search for the operator for each layer in a differentiable manner (Liu et al., 2018a; Wu et al., 2019).”
Benchmarking Deep Learning Models and Automated Model Design for COVID-19 Detection with Chest CT Scans (2020, methodology): “Many NAS studies [33], [34] use the MobileNetV2 structure to design the fac-”
AutoHAS: Differentiable Hyper-parameter and Architecture Search (2020, background): “2% Neural Architecture Search (NAS) has brought significant improvements in many applications, such as machine perception [19, 48, 5, 51, 50], language modeling [32, 11], and model compression [17, 10, 16].”
FBNetV3: Joint Architecture-Recipe Search using Neural Acquisition Function (2020, methodology): “One category of NAS is differentiable neural architecture search (DNAS).”
A Comprehensive Survey of Neural Architecture Search (2020, background): “architectures based on these parameter sharing often cannot be adequately trained, which will lead to inaccurate ranking of candidate network architectures. In NAS work based on gradient optimization [17, 102, 103], the joint optimization of supernet weights and architectural parameters will also in”
GreedyNAS: Towards Fast One-Shot NAS With Greedy Supernet (2020, methodology): “For example, DARTS [21] and its variants [38, 1] parameterize the supernet with an additional categorical distribution for indicating what operations we want to keep.”
BigNAS: Scaling Up Neural Architecture Search with Big Single-Stage Models (2020, methodology): “As a complementary direction, resource-aware NAS methods are proposed to simultaneously maximize prediction accuracy and minimize resource requirements such as latency, FLOPs, or memory footprints [4,9,30,31,33,34].”
Fast Neural Network Adaptation via Parameter Remapping and Architecture Search (2020, background): “As most differentiable NAS methods (Liu et al., 2019b; Cai et al., 2019; Wu et al., 2018) do, we relax every layer as a weighted sum of all candidate operations. where O denotes the operation set, α io denotes the architecture parameter of operation o in the i th layer, and x denotes the input…”
AtomNAS: Fine-Grained End-to-End Neural Architecture Search (2019, background): “Cai et al. (2019) and Wu et al. (2019) utilize supernets with multiple choices in each layer to accommodate a sampled network on the GPU.”
Memory-Efficient Hierarchical Neural Architecture Search for Image Denoising (2019, background): “For example, FBNet [38] andMNASNet [34]consider different expansion rates inside their modules to discover compact networks for image classification.”
Once for All: Train One Network and Specialize it for Efficient Deployment (2019, methodology): “0 317M 108ms 40, 000N - > 40K / 1,600K FBNet-C [27] 74.”
FairNAS: Rethinking Evaluation Fairness of Weight Sharing Neural Architecture Search (2019, methodology): “Many one-shot variants emerge and the design and training strategies of supernet can be roughly classified into three categories: training the whole supernet based on dropconnect tricks [2], jointly training the weights of choices and network parameters (in turns) [4, 11, 27], and training it in a s”
Single-Path NAS: Designing Hardware-Efficient ConvNets in less than 4 Hours (2019, methodology): “FBNet [21] searches on a “proxy” dataset (i.”
DetNAS: Backbone Search for Object Detection (2019, methodology): “Following works, including SNAS [27], Proxyless [3] and FBNet [26], also belong to one-shot NAS to some extent.”
Evaluating the Search Phase of Neural Architecture Search (2019, methodology): “…and performance predictors (Luo et al., 2018); 2) Exploiting NAS for other applications, e.g., object detection (Ghiasi et al., 2019; Chen et al., 2019), semantic segmentation (Liu et al., 2019a), and ﬁnding compact networks (Cai et al., 2018b; Wu et al., 2018; Chu et al., 2019; Guo et al., 2019).”