Designing Network Design Strategies Through Gradient Path Analysis

By Chien-Yao Wang, Hongpeng Liao, I-Hau Yeh. Journal of information science and engineering, 2022

377 citations recorded by the source corpus.

Designing a high-efficiency and high-quality expressive network architecture has always been the most important research topic in the field of deep learning. Most of today's network design strategies focus on how to integrate features extracted from different layers, and how to design computing units to effectively extract these features, thereby enhancing the expressiveness of the network. This paper proposes a new network design strategy, i.e., to design the network architecture based on gradient path analysis. On the whole, most of today's mainstream network design strategies are based on feed forward path, that is, the network architecture is designed based on the data path. In this paper, we hope to enhance the expressive ability of the trained model by improving the network learning ability. Due to the mechanism driving the network parameter learning is the backward propagation algorithm, we design network design strategies based on back propagation path. We propose the gradient path design strategies for the layer-level, the stage-level, and the network-level, and the design strategies are proved to be superior and feasible from theoretical analysis and experiments.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/95ef2ab2d37760d0e68155e43472c07d2aea3d59

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

YOLOv12: Attention-Centric Real-Time Object Detectors (2025, neutral): “YOLOv7 [57] introduce E-ELAN [56] (efficient layer aggregation networks) for improved gradient flow and various bag-of-freebies, while YOLOv8 [24] integrate a efficient C2f block for enhanced feature extraction.”
Hyper-YOLO: When Visual Object Detection Meets Hypergraph Computation (2024, neutral): “The architecture of YOLO consists of two main components: the backbone [7], [12], [13], [14] and neck [10], [15], [16].”
A Comprehensive Review of YOLO Architectures in Computer Vision: From YOLOv1 to YOLOv8 and YOLO-NAS (2023, methodology): “Inspired by GiraffeDet [112], CSPNet [69], and ELAN [108], the authors designed a neck that can work in real-time called Efficient-RepGFPN.”
YOLOv9: Learning What You Want to Learn Using Programmable Gradient Information (2024, methodology): “By combining two neural network architectures, CSPNet [64] and ELAN [65], which are designed with gradient path planning, we designed generalized efficient layer aggregation network (GELAN) that takes into account lighweight, inference speed, and accuracy.”
YOLOv10: Real-Time End-to-End Object Detection (2024, background): “It can serve as the efficient basic building block, e.g ., embedded in the ELAN structure [64, 21] (Fig.”