Interactive Perception: Leveraging Action in Perception and Perception in Action

By Jeannette Bohg, Karol Hausman, Bharathwaj Sankaran, Oliver Brock, Danica Kragic, Stefan Schaal, G. Sukhatme. IEEE Transactions on robotics, 2016

346 citations recorded by the source corpus.

Recent approaches in robot perception follow the insight that perception is facilitated by interaction with the environment. These approaches are subsumed under the term Interactive Perception (IP). This view of perception provides the following benefits. First, interaction with the environment creates a rich sensory signal that would otherwise not be present. Second, knowledge of the regularity in the combined space of sensory data and action parameters facilitates the prediction and interpretation of the sensory signal. In this survey, we postulate this as a principle for robot perception and collect evidence in its support by analyzing and categorizing existing work in this area. We also provide an overview of the most important applications of IP. We close this survey by discussing remaining open questions. With this survey, we hope to help define the field of Interactive Perception and to provide a valuable resource for future research.

Subfields: Cognitive Neuroscience, Control and Systems Engineering

View Paper: https://www.semanticscholar.org/paper/1fa9e326274b74aba5b9973ec057dd679cbab785

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Control and Systems Engineering

Citations in the corpus

Deep Object-Centric Representations for Generalizable Robot Learning (2017, background): “le.com/berkeley. edu/object-representations. II. RELATED WORK Vision for robotics is often approached differently from general computer vision as it involves interacting directly with the environment [12]. Robot perception is often concerned with object detection and localization at the instance lev”
SE3-nets: Learning rigid body motion using deep neural networks (2016, background): “interact with objects for manipulation or object exploration tasks, as summarized in a recent survey [12].”
3D Shape Perception from Monocular Vision, Touch, and Shape Priors (2018, neutral): “[2] for a thorough review.”
Learning to Singulate Objects using a Push Proposal Network (2017, background): “tion from interaction following the paradigm of interactive perception [2].”
Act the Part: Learning Interaction Strategies for Articulated Object Part Discovery (2021, positive): “Our approach builds on existing work in interactive perception [6], where visual tasks are solved using agent intervention.”
BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environments (2021, neutral): “…synchronized ground-truth information of the object being observed, its state and full shape. we expect the dataset to be useful to study and predict human gaze attention, and to develop new embodied AI algorithms for active [80, 81] and interactive perception [82]. and RL with motion primitives.”
Challenges and Outlook in Robotic Manipulation of Deformable Objects (2021, background): “A promising direction to deal with occlusion and noise is using active/interactive perception [38], [39].”
Trends and challenges in robot manipulation (2019, background): “Today, it is also recognized that perception and control are tightly coupled, and the field of interactive perception (63) regardsmanipulation as ameans to perceive and perception as a means to achieve better manipulation.”
Self-supervised 6D Object Pose Estimation for Robot Manipulation (2019, methodology): “By incorporating these techniques into the interactive robot perception framework [31], we can obtain accurate 6D poses of objects to improve the system.”
DREAM Architecture: a Developmental Approach to Open-Ended Learning in Robotics (2020, neutral): “Both steps are based on the interactive perception paradigm [67].”
A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithms (2019, background): “In interactive perception [33], the robot physically in-”
Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Tasks (2018, background): “There is compelling evidence that the interdependence and concurrency of different sensory streams aid perception and manipulation [9, 18, 31].”
Lessons from the Amazon Picking Challenge: Four Aspects of Building Robotic Systems (2016, neutral): “led to advances in robot perception [6, 27, 34].”
State Representation Learning for Control: An Overview (2018, background): “This is for instance the case of representation learning from diﬀerent temporal signals in [Duan, 2017, Bohg et al., 2017].”