Tree-Structured Reinforcement Learning for Sequential Object Localization

By Zequn Jie, Xiaodan Liang, Jiashi Feng, Xiaojie Jin, Wen Feng Lu, Shuicheng Yan. 2016

133 citations recorded by the source corpus.

Existing object proposal algorithms usually search for possible object regions over multiple locations and scales separately, which ignore the interdependency among different objects and deviate from the human perception procedure. To incorporate global interdependency between objects into object localization, we propose an effective Tree-structured Reinforcement Learning (Tree-RL) approach to sequentially search for objects by fully exploiting both the current observation and historical search paths. The Tree-RL approach learns multiple searching policies through maximizing the long-term reward that reflects localization accuracies over all the objects. Starting with taking the entire image as a proposal, the Tree-RL approach allows the agent to sequentially discover multiple objects via a tree-structured traversing scheme. Allowing multiple near-optimal policies, Tree-RL offers more diversity in search paths and is able to find multiple objects with a single feedforward pass. Therefore, Tree-RL can better cover different objects with various scales which is quite appealing in the context of object proposal. Experiments on PASCAL VOC 2007 and 2012 validate the effectiveness of the Tree-RL, which can achieve comparable recalls with current object proposal algorithms via much fewer candidate windows.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Glimpse Clouds: Human Activity Recognition from Unstructured Feature Points (2018, neutral): “Similar hard attention was used in multiple object recognition [2], object localization [7, 38, 23], saliency map generation [27], and action detection [60].”
Weakly Supervised Object Localization and Detection: A Survey (2021, unknown): “Several existing methods [11], [64], [74], [96], [192] have also demonstrated that designing deep reinforcement learning frameworks to model such a sequential searching process can indeed help to address the object localization, detection, and tracking problems in the computer vision community.”
From Discriminant to Complete: Reinforcement Searching-Agent Learning for Weakly Supervised Object Detection (2020, methodology): “Caicedo and Lazebnik [14] and Jie et al. [15] mine informative object proposals from each image via a DQN-based sequential search process.”
Partial Policy-Based Reinforcement Learning for Anatomical Landmark Localization in 3D Medical Images (2018, background): “[25] proposed a tree-structured RL with similar state and action representation.”
CIRL: Controllable Imitative Reinforcement Learning for Vision-based Self-driving (2018, methodology): “Deep-RL or RL algorithm has been applied to a wide variety of tasks, such as object recognition [21,22,23,24,25], computer games [26], robot locomotion [27], scene navigation [28] and autonomous driving in the simulators [29,6,30].”
AutoFocus: Efficient Multi-Scale Inference (2018, background): “AutoFocus shares the motivation with saliency and reinforcement learning based methods which perform a guided search while processing images [28, 26, 42, 53, 23, 49, 55, 29, 47], but it is designed to predict small objects in coarser scales and they need not be salient.”
Multi-Scale Deep Reinforcement Learning for Real-Time 3D-Landmark Detection in CT Scans (2019, methodology): “[60] proposed to use DRL for the sequential search of multiple objects.”
MASK-RL: Multiagent Video Object Segmentation Framework Through Reinforcement Learning (2020, methodology): “For these reasons, RL has recently attracted a lot of attention for emulating human behavior in several contexts, such as games [21], [29] and autonomous driving [22], but also for computer vision tasks [1], [13], [19], [25], [27], [47], including video object segmentation [11], [12], [37].”
Learning to Navigate for Fine-grained Classification (2018, background): “Navigating to possible informative regions can be viewed as a region proposal problem, which has been widely studied in [41,11,1,7,20].”
Temporally Grounding Natural Sentence in Video (2018, unknown): “Specifically, the task is usually formulated as a ranking problem over a set of candidate regions in a given image, where candidate spatial locations come from region proposal methods (Uijlings et al., 2013; Jie et al., 2016b,a; Ren et al., 2017) such as EdgeBox (Zitnick and Dollár, 2014).”
Deep Progressive Reinforcement Learning for Skeleton-Based Action Recognition (2018, unknown): “[35] to object detection.”
A Better Way to Attend: Attention With Trees for Video Question Answering (2018, methodology): “The tree structures have also been utilized for machine translation in [8] and for object localization in [14].”