More than a million ways to be pushed. A high-fidelity experimental dataset of planar pushing

By Kuan-Ting Yu, Maria Bauzá, Nima Fazeli, Alberto Rodriguez. 2016

199 citations recorded by the source corpus.

Pushing is a motion primitive useful to handle objects that are too large, too heavy, or too cluttered to be grasped. It is at the core of much of robotic manipulation, in particular when physical interaction is involved. It seems reasonable then to wish for robots to understand how pushed objects move. In reality, however, robots often rely on approximations which yield models that are computable, but also restricted and inaccurate. Just how close are those models? How reasonable are the assumptions they are based on? To help answer these questions, and to get a better experimental understanding of pushing, we present a comprehensive and high-fidelity dataset of planar pushing experiments. The dataset contains time-stamped poses of a circular pusher and a pushed object, as well as forces at the interaction. We vary the push interaction in 6 dimensions: surface material, shape of the pushed object, contact position, pushing direction, pushing speed, and pushing acceleration. An industrial robot automates the data capturing along precisely controlled position-velocity-acceleration trajectories of the pusher, which give dense samples of positions and forces of uniform quality. We finish the paper by characterizing the variability of friction, and evaluating the most common assumptions and simplifications made by models of frictional pushing in robotics.

Subfields: Biomedical Engineering, Control and Systems Engineering, Evaluation & Benchmarks, Robotics & Control

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biomedical Engineering

In Control and Systems Engineering

In Evaluation & Benchmarks

In Robotics & Control

Citations in the corpus

RoboNet: Large-Scale Multi-Robot Learning (2019, background): “Our work is inspired by this idea: while prior robotic manipulation methods and datasets [5, 46, 47, 26, 48, 7, 49] generally consider a single robot at a time, our dataset includes 7 different robots and data from 4 different institutions, with dozens of backgrounds and hundreds of viewpoints.”
Learning Synergies Between Pushing and Grasping with Self-Supervised Deep Reinforcement Learning (2018, background): “While inspiring, many of these methods rely on modeling assumptions that do not hold in practice [16], [4].”
Deep visual foresight for planning robot motion (2016, background): “Despite the extensive work in this area, tasks like pushing an unknown object to a desired position remain a challenging robotic task, largely due to the difficulties in estimating and modeling the physical world [10].”
Implicit Quantile Networks for Distributional Reinforcement Learning (2018, background): “As the agent interacts with the environment, irreducible randomness seeps in through the stochasticity of these interactions, the approximations in the agent’s representation, and even the inherently chaotic nature of physical interaction (Yu et al., 2016).”
IRIS: Implicit Reinforcement without Interaction at Scale for Learning Control from Offline Robot Manipulation Data (2019, background): “Recent research has successfully leveraged Reinforcement Learning (RL) for short-horizon robotic manipulation tasks, such as pushing and grasping objects [10, 22, 37].”
Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning (2019, background): “There is a long history of robotics benchmarks [28], datasets [29, 30, 31, 32, 33, 34, 35], competitions [36] and standardized object sets [37, 38] that have played an important role in robotics research.”
ROBOTURK: A Crowdsourcing Platform for Robotic Skill Learning through Imitation (2018, methodology): “Recent research has successfully employed Reinforcement Learning (RL), Self-Supervised Learning (SSL), and Imitation Learning (IL) for short-horizon skill learning, such as pushing and grasping objects [4, 5, 6].”
Modular meta-learning (2018, methodology): “(b) benchmarked robotic domains: MIT push dataset [24](top), action (throwing a ball) in the Berkeley MHAD dataset [25] (bottom)”
Realtime State Estimation with Tactile and Visual Sensing. Application to Planar Manipulation (2017, methodology): “[6] by analyzing a large experimental pushing dataset.”
RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control (2023, background): “We note that pushing dynamics are notoriously difficult to predict and control [88].”
Policy transfer via modularity (2017, background): “However, all these approaches make a number of assumptions about the friction that recent experiments have shown do not always hold in practice [10].”
π0: A Vision-Language-Action Flow Model for General Robot Control (2024, unknown): “Early work in this area often utilized self-supervised or autonomous data collection [26, 22, 8], providing a tractable data source for simple tasks such as grasping [18, 37] or pushing [56], but without the complexity of more dexterous behaviors.”
Learning Visual Affordances for Robotic Manipulation (2019, background): “While inspiring, many of these methods rely on modeling assumptions that do not hold in practice [120, 10].”
Benchmarking Simulated Robotic Manipulation Through a Real World Dataset (2019, background): “phasis is on tracking the effects of the robot in the context of a task, not the robot itself [22], [23].”
Learning to Singulate Objects using a Push Proposal Network (2017, background): “However, estimating the location of objects and other properties of the physical environment in order to plan push actions can be subject to errors, especially for unknown objects and cluttered scenes [21, 30, 31].”
A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithms (2019, methodology): “A grid approach may be used to ensure that the action samples are sufficiently spread out, but these approaches generally assume that the state can be reset between actions (Yu et al., 2016).”