Parametrized shape models for clothing

By Stephen Miller, Mario Fritz, Trevor Darrell, Pieter Abbeel. 2011

120 citations recorded by the source corpus.

We consider the problem of recognizing the configuration of clothing articles when crudely spread out on a flat surface, prior to and during folding. At the core of our approach are parametrized shape models for clothing articles. Each clothing category has its own shape model, and the variety in shapes for a given category is achieved through variation of the parameters. We present an efficient algorithm to find the parameters that provide the best fit when given an image of a clothing article. The models are such that, once the parameters have been fit, they provide a basic parse of the clothing article, allowing it to be followed by autonomous folding from category level specifications of fold sequences. Our approach is also able to recover the configuration of a clothing article when folds are being introduced—an important feature towards closing the perception-action loop. Additionally, our approach provides a reliable method of shape-based classification, simply by examining which model yields the best fit. Our experiments illustrate the effectiveness of our approach on a large set of clothing articles. Furthermore, we present an end-to-end system, which starts from an unknown spread-out clothing article, performs a parametrized model fit, then follows a category-level (rather than article specific) set of folding instructions, closing the loop with perceptual feedback by re-fitting between folds.

Subfields: Computational Mechanics, Computer Graphics and Computer-Aided Design, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Computer Graphics and Computer-Aided Design

In Computer Vision and Pattern Recognition

Citations in the corpus

SoftGym: Benchmarking Deep Reinforcement Learning for Deformable Object Manipulation (2020, methodology): “We use a GRU [2] with 200 hidden nodes as the deterministic path in the dynamics model.”
Combining self-supervised learning and imitation for vision-based rope manipulation (2017, background): “Perception of cloth and rope also poses a major challenge, since standard methods for estimating the pose of rigid objects cannot be readily applied to deformable objects for which it is difficult to concretely define the degrees of freedom or provide suitable training data [2].”
kPAM: KeyPoint Affordances for Category-Level Robotic Manipulation (2019, background): “the polygons in [17, 18, 28] on which the manipulation policy is defined.”
Parsing clothing in fashion photographs (2012, background): “Clothing items have also been used as implicit cues of identity in surveillance scenarios [26], to find people in an image collection of an event [11, 22, 25], to estimate occupation [23], or for robot manipulation [16].”
Tracking deformable objects with point clouds (2013, methodology): “[12] visually infer the state of a folded piece of cloth by optimizing a geometric model of folded clothing that mostly relies on the contour.”
KETO: Learning Keypoint Representations for Tool Manipulation (2019, background): “In robotic manipulation tasks with visual inputs, keypoints can provide compact information of the environment and the objects [11], [17], [54], [49], [33], [37], [34].”
A tutorial on task-parameterized movement learning and retrieval (2015, unknown): “Several denominations have been introduced in the literature to describe these models, such as task-parameterized [91, 63, 20] (the denomination used here), parametric [102, 51, 59], stylistic [13] or object-centric warping [56].”
Model for unfolding laundry using interactive perception (2011, background): “Folding, in particular, has received significant attention as of late [14] [15] [16].”