Bayesian Joint Modelling for Object Localisation in Weakly Labelled Images

By Zhiyuan Shi, Timothy M. Hospedales, Tao A. Xiang. 2015

31 citations recorded by the source corpus.

We address the problem of localisation of objects as bounding boxes in images and videos with weak labels. This weakly supervised object localisation problem has been tackled in the past using discriminative models where each object class is localised independently from other classes. In this paper, a novel framework based on Bayesian joint topic modelling is proposed, which differs significantly from the existing ones in that: (1) All foreground object classes are modelled jointly in a single generative model that encodes multiple object co-existence so that “explaining away” inference can resolve ambiguity and lead to better learning and localisation. (2) Image backgrounds are shared across classes to better learn varying surroundings and “push out” objects of interest. (3) Our model can be learned with a mixture of weakly labelled and unlabelled data, allowing the large volume of unlabelled images on the Internet to be exploited for learning. Moreover, the Bayesian formulation enables the exploitation of various types of prior knowledge to compensate for the limited supervision offered by weakly labelled data, as well as Bayesian domain adaptation for transfer learning. Extensive experiments on the PASCAL VOC, ImageNet and YouTube-Object videos datasets demonstrate the effectiveness of our Bayesian joint model for weakly supervised object localisation.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Learning and Refining of Privileged Information-Based RNNs for Action Recognition from Depth Sequences (2017, methodology): “Apart from optimizing network parameters, the bridging matrix of modeling latent PI can be updated iteratively by a closed-form solution in the M-step of EM procedure [31, 36, 3]:”
LayerCAM: Exploring Hierarchical Class Activation Maps for Localization (2021, background): “The effectiveness of the attention methods [1], [2] or attention modules [23], [24], [25] beneﬁt a lot of vision tasks [26], [27], [28], [29], [30], [31], [32].”
From Discriminant to Complete: Reinforcement Searching-Agent Learning for Weakly Supervised Object Detection (2020, background): “In early times, the learning methods are mainly designed based on hand-crafted principles and shallow learning models [23]–[27].”
Weakly Supervised Object Localization and Detection: A Survey (2021, unknown): “In [125], [126], Shi et al. explore the appearance prior and geometry prior in their topic model to build a Bayesian joint modeling framework for weakly supervised object localization.”
Weakly Supervised Object Localization Using Things and Stuff Transfer (2017, unknown): “We quantify localization performance in the target training set with the CorLoc measure [9, 4, 10, 50, 28, 27].”
PCL: Proposal Cluster Learning for Weakly Supervised Object Detection (2018, unknown): “Many methods are emerging for the WSOD problem [13], [14], [33], [34], [35], [36], [37], [38], [39], [40], [58], [59], [60], [61].”