Bottom-Up and Top-Down Reasoning with Hierarchical Rectified Gaussians

By Peiyun Hu, Deva Ramanan. 2015

115 citations recorded by the source corpus.

Convolutional neural nets (CNNs) have demonstrated remarkable performance in recent history. Such approaches tend to work in a "unidirectional" bottom-up feed-forward fashion. However, practical experience and biological evidence tells us that feedback plays a crucial role, particularly for detailed spatial understanding tasks. This work explores "bidirectional" architectures that also reason with top-down feedback: neural units are influenced by both lower and higher-level units. We do so by treating units as rectified latent variables in a quadratic energy function, which can be seen as a hierarchical Rectified Gaussian model (RGs) [39]. We show that RGs can be optimized with a quadratic program (QP), that can in turn be optimized with a recurrent neural network (with rectified linear units). This allows RGs to be trained with GPU-optimized gradient descent. From a theoretical perspective, RGs help establish a connection between CNNs and hierarchical probabilistic models. From a practical perspective, RGs are well suited for detailed spatial tasks that can benefit from top-down reasoning. We illustrate them on the challenging task of keypoint localization under occlusions, where local bottom-up evidence may be misleading. We demonstrate state-of-the-art results on challenging benchmarks.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Probabilistic & Bayesian Methods

Citations in the corpus

Perceiver: General Perception with Iterative Attention (2021, background): “Re-entrant processing of this kind (sometimes referred to as top-down processing) has a long history in computer vision (Borenstein et al., 2004; Kumar et al., 2005; Carreira et al., 2016; Hu & Ramanan, 2016; Yang et al., 2018; Lin et al., 2020).”
Stacked Hourglass Networks for Human Pose Estimation (2016, background): “Hu and Ramanan [27] have an architecture more similar to ours that can also be used for multiple stages of predictions, but their model ties weights in the bottom-up and top-down portions of computation as well as across iterations.”
Deep High-Resolution Representation Learning for Visual Recognition (2019, neutral): “We review closely-related representation learning techniques developed mainly for human pose estimation [57], semantic segmentation and object detection, from three aspects: low-resolution representation learning, highresolution representation recovering, and high-resolution representation maintaini”
Deep High-Resolution Representation Learning for Human Pose Estimation (2019, background): “The high-to-low process aims to generate low-resolution and high-level representations, and the low-to-high process aims to produce highresolution representations [4, 11, 22, 70, 39, 60].”
Associative Embedding: End-to-End Learning for Joint Detection and Grouping (2016, background): “Multiperson Pose Estimation Recent methods have made great progress improving human pose estimation in images in particular for single person pose estimation [50, 48, 52, 40, 8, 5, 41, 4, 14, 19, 34, 26, 7, 49, 44].”
Dynamic Computational Time for Visual Attention (2017, background): “Hu and Ramanan [38] show that network feedbacks can help better locating human face landmarks.”
Chained Predictions Using Convolutional Neural Networks (2016, unknown): “0 Hu&Ramanan [38] 95.”
Monocular 3D Human Pose Estimation Using Transfer Learning and Improved CNN Supervision (2016, methodology): “In 2D joint detection and pose estimation from RGB, data-driven approaches using Convolutional Neural Networks (CNNs) have shown impressive results [18, 82, 78, 79, 28, 51, 14, 9, 47, 42, 15, 25, 27], outperforming previous hand crafted and model-based methods by a large margin [1, 5, 22].”
Self Adversarial Training for Human Pose Estimation (2017, unknown): “[18], CVPR’16 95.”
Multigrid Neural Architectures (2016, background): “These include skip connections and upsampling [24, 3], hypercolumns [10], and, autoencoder-like, hourglass or U-shaped networks that reduce and re-expand spatial grids [1, 13, 27, 30].”
Survey on Emotional Body Gesture Recognition (2018, background): “estimation from single monocular images [39], [40], [41], [42], [72], [73], [74], [75] and from videos [76], or multi-person pose estimation [77], [78], [79] and 3D pose estimation [80] are all”
Multi-context Attention for Human Pose Estimation (2017, unknown): “0 Hu&Ramanan [21] 95.”
PoseTrack: A Benchmark for Human Pose Estimation and Tracking (2017, background): “Human pose estimation has recently made significant progress on the tasks of single person pose estimation in individual frames [46, 45, 44, 4, 49, 15, 18, 31, 2, 36] and videos [34, 6, 21, 12] as well as multi-person pose estimation in monocular images [35, 18, 20, 3, 32].”
Learning Feature Pyramids for Human Pose Estimation (2017, unknown): “0 Hu&Ramanan [28] 95.”
Jointly Optimize Data Augmentation and Network Training: Adversarial Data Augmentation in Human Pose Estimation (2018, background): “With recent advances in Deep Neural Networks (DNNs), image based human pose estimation has achieved significant progress in the past few years [4, 35, 15, 28, 22, 11, 17, 40, 3, 24].”