Learning Semantic Embedding Spaces for Slicing Vegetables

By Mohit Sharma, Kevin Zhang, Oliver Kroemer. 2019

15 citations recorded by the source corpus.

In this work, we present an interaction-based approach to learn semantically rich representations for the task of slicing vegetables. Unlike previous approaches, we focus on object-centric representations and use auxiliary tasks to learn rich representations using a two-step process. First, we use simple auxiliary tasks, such as predicting the thickness of a cut slice, to learn an embedding space which captures object properties that are important for the task of slicing vegetables. In the second step, we use these learned latent embeddings to learn a forward model. Learning a forward model affords us to plan online in the latent embedding space and forces our model to improve its representations while performing the slicing task. To show the efficacy of our approach we perform experiments on two different vegetables: cucumbers and tomatoes. Our experimental evaluation shows that our method is able to capture important semantic properties for the slicing task, such as the thickness of the vegetable being cut. We further show that by using our learned forward model, we can plan for the task of vegetable slicing.

Subfields: Control and Systems Engineering, Plant Science, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Control and Systems Engineering

  1. Kristen Grauman
  2. Kris M. Kitani
  3. Tushar Nagarajan
  4. James M. Rehg
  5. Miao Liu
  6. Dhruv Batra
  7. Hanbyul Joo
  8. Giovanni Maria Farinella

In Plant Science

  1. Lorenzo Torresani

In Representation & Self-Supervised Learning

  1. Kristen Grauman
  2. Antonio Torralba
  3. Jitendra Malik
  4. Andrea Vedaldi
  5. Christoph Feichtenhofer
  6. Aude Oliva
  7. Lorenzo Torresani
  8. Rohit Girdhar

Citations in the corpus