Learning Visual Affordances for Robotic Manipulation

By Andy Zeng. 2019

22 citations recorded by the source corpus.

Subfields: Computer Vision and Pattern Recognition, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

1. Luc Van Gool
2. Jitendra Malik
3. Li Fei-Fei
4. Dacheng Tao
5. Trevor Darrell
6. Xiaogang Wang
7. Ming-Hsuan Yang
8. Kristen Grauman

In Reinforcement Learning

1. Andy Zeng
2. Shuran Song
3. Johnny Lee
4. Karol Hausman
5. Brian Ichter
6. Peter R. Florence
7. Wolfram Burgard
8. Oier Mees

Citations in the corpus

• Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language (2022, background): “These ideas may give rise to a new class of robot systems where by grounding affordances (Zeng 2019) on language, control algorithms can begin to tap into the capabilities of models trained on Internet-scale data, and to tackle applications that have traditionally been data-scarce.”
• TossingBot: Learning to Throw Arbitrary Objects With Residual Physics (2019, methodology): “sping success (i.e., grasping affordance) when executing a top-down parallel-jaw grasp centered at the 3D location of i2I with the gripper oriented horizontally with respect to the heightmap I. As in [42,41,39,38], we account for different grasping angles by rotating the input heightmap by 16 orient”
• Visual Language Maps for Robot Navigation (2022, methodology): “In contrast to prior work, which may reference these subgoals with language and map to low-level policies with semantic translation [39] or affordances [35], [40], [41], [42], we leverage the code-writing capabilities of LLMs to generate executable Python robot code [43], [33], [44], [27] that can (”
• Code as Policies: Language Model Programs for Embodied Control (2022, methodology): “decompose natural language commands into sequences of executable actions by text completion and semantic translation [14], while SayCan [17] generates feasible plans for robots by jointly decoding an LLM weighted by skill affordances [20] from value functions.”
• Spatial Action Maps for Mobile Manipulation (2020, background): “…segmentation [14] has been shown to benefit from pixel-aligned input and output representations, while end-to-end robotic manipulation [21, 45, 44, 43] is significantly more sample efficient when using FCNs to predict state-action values for a dense sampling of grasps aligned with the visual input.”
• Clear Grasp: 3D Shape Estimation of Transparent Objects for Manipulation (2019, methodology): “We adapted a state-of-the-art grasping algorithm for our experiment [60, 59, 61, 58, 55], which consists of a convolutional network that predicts the probability of picking success for a scripted grasping primitive across a dense pixelwise sampling of end effector locations and orientations across t”
• Learning to Rearrange Deformable Cables, Fabrics, and Bags with Goal-Conditioned Transporter Networks (2020, unknown): “This technique [67] has shown promising results for robotic warehouse order picking [71], pushing and grasping [70], tossing objects [69], kit assembly [66], and mobile manipulation [62].”