Multi-scale Recognition with DAG-CNNs

By Songfan Yang, Deva Ramanan. 2015

209 citations recorded by the source corpus.

We explore multi-scale convolutional neural nets (CNNs) for image classification. Contemporary approaches extract features from a single output layer. By extracting features from multiple layers, one can simultaneously reason about high, mid, and low-level features during classification. The resulting multi-scale architecture can itself be seen as a feed-forward model that is structured as a directed acyclic graph (DAG-CNNs). We use DAG-CNNs to learn a set of multi-scale features that can be effectively shared between coarse and fine-grained classification tasks. While fine-tuning such models helps performance, we show that even "off-the-self" multi-scale features perform quite well. We present extensive analysis and demonstrate state-of-the-art classification performance on three standard scene benchmarks (SUN397, MIT67, and Scene15). In terms of the heavily benchmarked MIT67 and Scene15 datasets, our results reduce the lowest previously-reported error by 23.9% and 9.5%, respectively.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Growing a Brain: Fine-Tuning by Increasing Model Capacity (2017, methodology): “For a fair comparison, we focus on the approaches that use single scale AlexNet CNNs. Importantly, our approach can be also combined with other CNN variations ( e.g ., VGG-CNN [37], multi-scale CNN [12, 53]) for further improvement.”
CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification (2021, methodology): “In the context of CNNs, multi-scale feature representations have been used for detection and recognition of objects at multiple scales [4, 22, 44, 26], as well as to speed up neural networks in Big-Little Net [5] and OctNet [6].”
Laplacian Reconstruction and Refinement for Semantic Segmentation (2016, methodology): “Performing classiﬁcation using a “jet” of feature map responses aggregated across multiple layers has been successfully leveraged for semantic segmentation [5], generic boundary detection [20], simultaneous detection and segmentation [4], and scene recognition [21].”
Convolutional Networks with Dense Connectivity (2019, background): “Several recent publications [14], [15], [16], [17], have found the utilization of multi-level features in CNNs through skip-connections effective for various vision tasks.”
Scene Recognition with CNNs: Objects, Scales and Dataset Bias (2016, unknown): “11 DAG-CNN[20] IN 1 - 92.”
Learning Short-Cut Connections for Object Counting (2018, background): “In recent years, the benefits of short-cut connections in deep neural architectures have been demonstrated in various empirical studies [5, 14, 15, 16, 34, 42].”
Laplacian Pyramid Reconstruction and Refinement for Semantic Segmentation (2016, methodology): “Performing classification using a “jet” of feature map responses aggregated across multiple layers has been successfully leveraged for semantic segmentation [24], generic boundary detection [32], simultaneous detection and segmentation [15], and scene recognition [33].”
AOGNets: Deep AND-OR Grammar Networks for Visual Recognition (2017, methodology): “connection adapt the sequential architecture to directed and acyclic graph (DAG) structured networks, which were explored earlier in the context of recurrent neural networks (RNN) [3,18] and ConvNets [67]. Most work focused on boosting spatial encoding and utilizing spatial dimensionality reduction.”
Visual Attention Methods in Deep Learning: An In-Depth Survey (2022, background): “The module, named Higher-Order Group Attention (HOGA), is in the form of a Directed Acyclic Graph (DAG) [141, 183, 162, 172] where each node represents a group and each edge represents a heterogeneous attention operation.”
Knowledge Guided Disambiguation for Large-Scale Scene Classification With Multi-Resolution CNNs (2016, background): “SCENE recognition [1], [2] is a fundamental problem in computer vision, and has received increasing attention in the past few years [3], [4], [5], [6], [7], [8], [9], [10], [11].”
AOGNets: Compositional Grammatical Architectures for Deep Learning (2017, methodology): “connection adapt the sequential architecture to directed and acyclic graph (DAG) structured networks, which were explored earlier in the context of recurrent neural networks (RNN) [3,18] and ConvNets [67]. Most work focused on boosting spatial encoding and utilizing spatial dimensionality reduction.”
Learning Deep Structured Multi-Scale Features using Attention-Gated CRFs for Contour Prediction (2017, background): “Yang et al. [43] proposed DAG-CNNs where multi-scale feature outputs from different ReLU layers are combined through element-wise addition operator.”
HyperFace: A Deep Multi-Task Learning Framework for Face Detection, Landmark Localization, Pose Estimation, and Gender Recognition (2019, background): “Yang and Ramanan [61] proposedDAG-CNNs, which extract features from multiple layers to reason about high, mid and low-level fea-”
HiFormer: Hierarchical Multi-scale Representations Using Transformers for Medical Image Segmentation (2022, background): “[38, 53, 9, 34], PVT [48] was the first introduced pyramid vision transformer.”