Visual closed-loop control for pouring liquids

By Connor Schenck, Dieter Fox. 2016

116 citations recorded by the source corpus.

Pouring a specific amount of liquid is a challenging task. In this paper we develop methods for robots to use visual feedback to perform closed-loop control for pouring liquids. We propose both a model-based and a model-free method utilizing deep learning for estimating the volume of liquid in a container. Our results show that the model-free method is better able to estimate the volume. We combine this with a simple PID controller to pour specific amounts of liquid, and show that the robot is able to achieve an average 38ml deviation from the target amount. To our knowledge, this is the first use of raw visual feedback to pour liquids in robotics.

Subfields: Aerospace Engineering, Biomedical Engineering, Control and Systems Engineering

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Biomedical Engineering

In Control and Systems Engineering

Citations in the corpus

End-to-End Robotic Reinforcement Learning without Reward Engineering (2019, background): “For example, using thermal cameras for tracking fluids [35], mocap sensors [21] or computer vision systems [34] for tracking objects, and accelerometers for determining the state of a door [44].”
Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translation (2017, methodology): “Besides [9], tasks involving pouring were also studied in [31] using a simple”
Hierarchical Foresight: Self-Supervised Learning of Long-Horizon Tasks via Visual Subgoal Generation (2019, background): “…complex tasks from pixels (Ghadirzadeh et al., 2017; Levine et al., 2015; Kalashnikov et al., 2018; Lange et al., 2012; OpenAI et al., 2018; Schenck & Fox, 2016; Matas et al., 2018; James et al., 2017; Singh et al., 2019), including in goal-conditioned settings (Kaelbling, 1993; Schaul et…”
PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-training (2021, background): “…application area in which HiL methods could have signiﬁcant real-world impact is robotic manipulation, since learning often requires extensive engineering in the real world (Yahya et al., 2017; Schenck & Fox, 2017; Kormushev et al., 2010; Rusu et al., 2017; Akkaya et al., 2019; Peng et al., 2020).”
AVID: Learning Multi-Stage Tasks via Pixel-Level Translation of Human Videos (2019, background): “typically requires highly instrumented, and sometimes creative, setups such as motion capture and object tracking [13], [14], accelerometers to detect door opening [15], or thermal cameras to detect pouring [16].”
COG: Connecting New Skills to Past Experience with Offline Reinforcement Learning (2020, background): “Robotic RL. RL has been applied to a wide variety of robotic manipulation tasks, including grasping objects [11, 12], in-hand object manipulation [13, 14, 15, 16], pouring ﬂuids [17], door opening [18, 19], and manipulating cloth [20, 21].”
The Ingredients of Real-World Robotic Reinforcement Learning (2020, background): “…to detect objects and extract the reward signals (Devin et al., 2018; Nagabandi et al., 2019), engineer reward functions that use various task-speciﬁc heuristics to obtain rewards from pixels (Schenck & Fox, 2017; Kalashnikov et al., 2018), or instrument every environment (Chebotar et al., 2017).”
Few-Shot Goal Inference for Visuomotor Learning and Planning (2018, background): “Robotic learning methods often sidestep this issue and instead hand-engineer task objectives [15, 16], often using manually instrumented environments (e.g., a thermal camera or scale to evaluate pouring [17, 18, 19], mocap to track a pancake [20], and an accelerometer on a door handle [21]).”
Reasoning About Liquids via Closed-Loop Simulation (2017, background): “Several others have also performed the pouring task using a real robot [25, 13, 18, 29, 3, 22].”