StarMap for Category-Agnostic Keypoint and Viewpoint Estimation

By Xingyi Zhou, Arjun Karpur, Linjie Luo, Qi-Xing Huang. 2018

93 citations recorded by the source corpus.

Semantic keypoints provide concise abstractions for a variety of visual understanding tasks. Existing methods define semantic keypoints separately for each category with a fixed number of semantic labels in fixed indices. As a result, this keypoint representation is in-feasible when objects have a varying number of parts, e.g. chairs with varying number of legs. We propose a category-agnostic keypoint representation, which combines a multi-peak heatmap (StarMap) for all the keypoints and their corresponding features as 3D locations in the canonical viewpoint (CanViewFeature) defined for each instance. Our intuition is that the 3D locations of the keypoints in canonical object views contain rich semantic and compositional information. Using our flexible representation, we demonstrate competitive performance in keypoint detection and localization compared to category-specific state-of-the-art methods. Moreover, we show that when augmented with an additional depth channel (DepthMap) to lift the 2D keypoints to 3D, our representation can achieve state-of-the-art results in viewpoint estimation. Finally, we show that our category-agnostic keypoint representation can be generalized to novel categories.

Subfields: Computer Vision and Pattern Recognition, 3D Vision & Pose

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In 3D Vision & Pose

Citations in the corpus

Objects as Points (2019, methodology): “Object detection is then a standard keypoint estimation problem [3,39,60].”
Bottom-Up Object Detection by Grouping Extreme and Center Points (2019, methodology): “StarMap [53] mixes all types of keypoints using a single heatmap for general keypoint detection.”
HybridPose: 6D Object Pose Estimation Under Hybrid Representations (2020, background): “A common approach represents keypoints as peaks of heatmaps [28, 48], which becomes sub-optimal when keypoints are occluded, as the input image does not provide explicit visual cues for their locations.”
CDPN: Coordinates-Based Disentangled Pose Network for Real-Time RGB-Based 6-DoF Object Pose Estimation (2019, background): “[19, 29] predicted semantic keypoints for pose and viewpoint estimation with an hourglassbased network [17, 5].”