GhostNet: More Features From Cheap Operations

By Kai Han, Yunhe Wang, Qi Tian, Jianyuan Guo, Chunjing Xu. 2019

4,183 citations recorded by the source corpus.

Deploying convolutional neural networks (CNNs) on embedded devices is difficult due to the limited memory and computation resources. The redundancy in feature maps is an important characteristic of those successful CNNs, but has rarely been investigated in neural architecture design. This paper proposes a novel Ghost module to generate more feature maps from cheap operations. Based on a set of intrinsic feature maps, we apply a series of linear transformations with cheap cost to generate many ghost feature maps that could fully reveal information underlying intrinsic features. The proposed Ghost module can be taken as a plug-and-play component to upgrade existing convolutional neural networks. Ghost bottlenecks are designed to stack Ghost modules, and then the lightweight GhostNet can be easily established. Experiments conducted on benchmarks demonstrate that the proposed Ghost module is an impressive alternative of convolution layers in baseline models, and our GhostNet can achieve higher recognition performance (e.g. 75.7% top-1 accuracy) than MobileNetV3 with similar computational cost on the ImageNet ILSVRC-2012 classification dataset. Code is available at https://github.com/huawei-noah/ghostnet.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Radiology, Nuclear Medicine and Imaging

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Radiology, Nuclear Medicine and Imaging

Citations in the corpus

AutoGAN-Distiller: Searching to Compress Generative Adversarial Networks (2020, neutral): “With the unprecedented demand for deploying deep models on resource-constrained platforms, many compression methods are proposed to narrow the gap between the model complexity and on-device resources (Han et al., 2020; Wang et al., 2018b; Chen et al., 2020a).”
Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet (2021, methodology): “Speciﬁcally, we investigate Wide-ResNets (shallow-wide vs deep-narrow structure) [52], DenseNet (dense connection) [21], ResneXt structure [44], Ghost operation [14, 59] and channel attention [20].”
YOLOv4: Optimal Speed and Accuracy of Object Detection (2020, methodology): “• For VPU - we use grouped-convolution, but we refrain from using Squeeze-and-excitement (SE) blocks - specifically this includes the following models: EfficientNet-lite / MixNet [76] / GhostNet [21] / MobileNetV3”
CMT: Convolutional Neural Networks Meet Vision Transformers (2021, background): “tectural advances, there has also been works [13, 24, 39] focusing on improving over-parameterized deep neural networks by trading accuracy for efficiency.”
Rewrite the Stars (2024, neutral): “These include depth-wise convolution [28, 46], feature re-use [6, 19], and re-parameterization [55], among others.”
SCSA: Exploring the Synergistic Effects Between Spatial and Channel Attention (2024, neutral): “2.”
UAV-YOLOv8: A Small-Object-Detection Model Based on Improved YOLOv8 for UAV Aerial Photography Scenarios (2023, neutral): “Some classical lightweight networks, such as MobileNet [43], ShufﬂeNet [44], and GhostNet [45], are using deep convolution or group convolution to extract the spatial features of images.”
Run, Don't Walk: Chasing Higher FLOPS for Faster Neural Networks (2023, methodology): “MobileNets [24, 25, 54], ShuffleNets [46, 84] and GhostNet [17], among others, leverage the depthwise convolution (DWConv) [55] and/or group convolution (GConv) [31] to extract spatial features.”
TopFormer: Token Pyramid Transformer for Mobile Semantic Segmentation (2022, methodology): “GhostNet [13] uses the cheaper operator, depth-wise convolutions, to generate more features.”
Distilling Object Detectors via Decoupled Features (2021, background): “Techniques such as quantization [19, 58, 31, 57, 62], pruning [2, 17, 20], network design [55, 49, 15, 18] and knowledge distillation [56, 6] have been developed to overcome this dilemma and achieve an efficient inference on detection task.”
AdaFuse: Adaptive Temporal Fusion Network for Efficient Action Recognition (2021, methodology): “In particular, orthogonal to the design of compact models, the computational cost of a CNN model also has much to do with the redundancy of CNN features (Han et al., 2019).”
CABNet: Category Attention Block for Imbalanced Diabetic Retinopathy Grading (2020, background): “However, some channels in existing CNN models either lack the inter-class diversity, or are redundant in the feature maps [13], which will limit the representational ability of CNNs.”
EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attention (2023, background): “With the demand of deploying CNNs on resource-constrained scenarios, efficient CNNs have been intensively studied in literature [23, 24, 26, 48, 63, 67, 87].”
Towards Real-World Blind Face Restoration with Generative Facial Prior (2021, methodology): “MobileNet [29] proposes depthwise convolutions and GhostNet [24] splits the convolutional layer into two parts and uses fewer filters to generate intrinsic feature maps.”
CARS: Continuous Evolution for Efficient Neural Architecture Search (2019, background): “Convolutional neural networks have made great progress in a large range of computer vision tasks, such as recognition [19, 17, 16], detection [11], and segmentation [18].”