Zero-shot Learning of 3D Point Cloud Objects

By Ali Cheraghian, Shafin Rahman, Lars Petersson. 2019

66 citations recorded by the source corpus.

Recent deep learning architectures can recognize instances of 3D point cloud objects of previously seen classes quite well. At the same time, current 3D depth camera technology allows generating/segmenting a large amount of 3D point cloud objects from an arbitrary scene, for which there is no previously seen training data. A challenge for a 3D point cloud recognition system is, then, to classify objects from new, unseen, classes. This issue can be resolved by adopting a zero-shot learning (ZSL) approach for 3D data, similar to the 2D image version of the same problem. ZSL attempts to classify unseen objects by comparing semantic information (attribute/word vector) of seen and unseen classes. Here, we adapt several recent 3D point cloud recognition systems to the ZSL setting with some changes to their architectures. To the best of our knowledge, this is the first attempt to classify unseen 3D point cloud objects in the ZSL setting. A standard protocol (which includes the choice of datasets and the seen/unseen split) to evaluate such systems is also proposed. Baseline performances are reported using the new protocol on the investigated models. This investigation throws a new challenge to the 3D point cloud recognition community that may instigate numerous future works.

Subfields: Aerospace Engineering, Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Semantic-aware Knowledge Distillation for Few-Shot Class-Incremental Learning (2021, background): “Being inspired by the literature on zero-shot learning [7, 6, 5, 20, 37, 24, 25], given an image as input, we estimate the semantic word vectors for the input instead of directly predicting its class label.”
PointCLIP: Point Cloud Understanding by CLIP (2021, background): “By leaning a projection function from point cloud feature space to the category semantic space, [7] trains PointNet [44] by the former and tests it on the latter.”
Mitigating the Hubness Problem for Zero-Shot Learning of 3D Objects (2019, methodology): “As shown, seen and unseen, and HM accuracy are improved using our proposed method for both w2v semantics and GloVe.”
Transductive Zero-Shot Learning for 3D Point Cloud Classification (2019, methodology): “For the experiments on 3D data, we compare with two 3D ZSL methods, ZSLPC [9] and MHPC [8], and three 2D ZSL methods, f-CLSWGAN [58], CADA-VAE [43], and QFSL [49].”