Representation and recognition of the spatial organization of three-dimensional shapes

By D. Marr, H. Nishihara. 1978

The human visual process can be studied by examining the computational problems associated with deriving useful information from retinal images. In this paper, we apply this approach to the problem of representing three-dimensional shapes for the purpose of recognition. 1. Three criteria, accessibility, scope and uniqueness, and stability and sensitivity, are presented for judging the usefulness of a representation for shape recognition. 2. Three aspects of a representation’s design are considered, (i) the representation’s coordinate system, (ii) its primitives, which are the primary units of shape information used in the representation, and (iii) the organization the representation imposes on the information in its descriptions. 3. In terms of these design issues and the criteria presented, a shape representation for recognition should: (i) use an object-centred coordinate system, (ii) include volumetric primitives of varied sizes, and (iii) have a modular organization. A representation based on a shape’s natural axes (for example the axes identified by a stick figure) follows directly from these choices. 4. The basic process for deriving a shape description in this representation must involve: (i) a means for identifying the natural axes of a shape in its image and (ii) a mechanism for transforming viewer-centred axis specifications to specifications in an object-centred coordinate system. 5. Shape recognition involves: (i) a collection of stored shape descriptions, and (ii) various indexes into the collection that allow a newly derived description to be associated with an appropriate stored description. The most important of these indexes allows shape recognition to proceed conservatively from the general to the specific based on the specificity of the information available from the image. 6. New constraints supplied by a conservative recognition process can be used to extract more information from the image. A relaxation process for carrying out this constraint analysis is described.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Lifting Object Detection Datasets into 3D (2015, background): “Instead, their model searches the training set for patches similar to those in the test image, then transfers the associated depth information.”
Part-Based Representations of Visual Shape and Implications for Visual Cognition (2001, background): “From a computational perspective, computed parts must satisfy certain principles in order to be useful (Sutherland, 1968; Marr & Nishihara, 1978; Hoffman & Richards, 1984).”
Unite the People: Closing the Loop Between 3D and 2D Human Representations (2017, background): “Body representations beyond 3D skeletons have a long history in the computer vision community [17, 30, 31, 35].”
The Invariance Hypothesis Implies Domain-Specific Regions in Visual Cortex (2014, background): “[49] D Marr and Herbert Keith Nishihara.”
Three learning phases for radial-basis-function networks (2001, unknown): “…Poggio & Edelman, 1990; Schiele & Crowley, 1996), computer vision and pattern recognition (Basri, 1996; Edelman & Duvdevani-Bar , 1997; Marr, 1982; Marr & Nishihara, 1978; Ullmann, 1996), psychophysics and brain theory (Bu Èlthoff, Edelman, & Tarr, 1995; Edelman & Bu Èlthoff, 1992; Logothetis…”
DepthSynth: Real-Time Realistic Synthetic Data Generation from CAD Models for 2.5D Recognition (2017, background): “Synthetic Depth Image Generation Early research along this direction involves the work of [29, 5], wherein search based on 3D representations are introduced.”
Teaching 3D geometry to deformable part models (2012, background): “3D geometric object class representations have been considered the holy grail of computer vision since its early days [23] [5] [26] [22], but proved difficult to match robustly to real world images.”
Cognitive Perspectives in Psychology (2013, unknown): “The functions Marr developed were mathematical models of such processes as edge detection, the perception of shapes at different scales, and stereopsis (Marr & Nishihara, 1978).”
Retrieving Articulated 3-D Models Using Medial Surfaces and Their Graph Spectra (2005, background): “The computer vision community has grappled with the problem of generic level object recognition by suggesting representations based on volumetric parts, including generalized cylinders, superquadrics and geons [5, 16 , 19, 4]. Such approaches build a degree of robustness to deformations and movemen”
Deformable M-Reps for 3D Medical Image Segmentation (2003, methodology): “The relation has also been explored in 3D by Nackman [1985], Vermeer [1994], and Siddiqi [1999], and medial axis modeling techniques have been applied by many researchers, including Bloomenthal [1991], Wyvill [1986], Singh [1998], Amenta [1998], Bittar [1995], Igarashi [1999] and Markosian [1999].”
Dark, Beyond Deep: A Paradigm Shift to Cognitive AI with Humanlike Common Sense (2020, background): “5D [196] sketches; and (iii) a high-level vision system in charge of full 3D scene formation [197, 198, 199].”
Transformation of shape information in the ventral pathway. (2007, background): “com Structural representation in object-based coordinates Many theories of shape processing [27,28] are based on the idea of structural representation — that is, shape description in terms of object parts and their positional and connectional relationships.”
Detailed 3D Representations for Object Recognition and Modeling (2013, background): “Starting from Marr's seminal ideas [39], many 3D models of objects were proposed which provided rich and detailed descriptions of object shape and pose [8], [44], [38], [27], [53], [22].”
3D Face Morphable Models "In-the-Wild" (2017, methodology): “As a set of four illumination conditions are provided for each subject then we can generate ground-truth facial surface normals using calibrated 4-source Photometric Stereo [27].”