Realtime State Estimation with Tactile and Visual Sensing. Application to Planar Manipulation

By Kuan-Ting Yu, Alberto Rodriguez. 2017

42 citations recorded by the source corpus.

Accurate and robust object state estimation enables successful object manipulation. Visual sensing is widely used to estimate object poses. However, in a cluttered scene or in a tight workspace, the robot's end-effector often occludes the object from the visual sensor. The robot then loses visual feedback and must fall back on open-loop execution. In this paper, we integrate both tactile and visual input using a framework for solving the SLAM problem, incremental smoothing and mapping (iSAM), to provide a fast and flexible solution. Visual sensing provides global pose information but is noisy in general, whereas contact sensing is local, but its measurements are more accurate relative to the end-effector. By combining them, we aim to exploit their advantages and overcome their limitations. We explore the technique in the context of a pusher-slider system. We adapt iSAM's measurement cost and motion cost to the pushing scenario, and use an instrumented setup to evaluate the estimation quality with different object shapes, on different surface materials, and under different contact modes.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation (2018, background): “While most prior grasping methods operate in open-loop, a number of works have studied closedloop grasping [22, 23, 24, 25].”
How to train your robot with deep reinforcement learning: lessons we have learned (2021, background): “A number of works have studied closed loop grasping (Yu and Rodriguez 2018; Viereck et al. 2017; Hausman et al. 2017; Levine et al. 2018).”