Mirror, mirror on the wall, tell me, is the error small?

By Heng Yang, Ioannis Patras. 2015

51 citations recorded by the source corpus.

Do object part localization methods produce bilaterally symmetric results on mirror images? Surprisingly not, even though state of the art methods augment the training set with mirrored images. In this paper we take a closer look into this issue. We first introduce the concept of mirrorability as the ability of a model to produce symmetric results in mirrored images and introduce a corresponding measure, namely the mirror error that is defined as the difference between the detection result on an image and the mirror of the detection result on its mirror image. We evaluate the mirrorability of several state of the art algorithms in two of the most intensively studied problems, namely human pose estimation and face alignment. Our experiments lead to several interesting findings: 1) Most of state of the art methods struggle to preserve the mirror symmetry, despite the fact that they do have very similar overall performance on the original and mirror images; 2) the low mirrorability is not caused by training or testing sample bias - all algorithms are trained on both the original images and their mirrored versions; 3) the mirror error is strongly correlated to the localization/alignment error (with correlation coefficients around 0.7). Since the mirror error is calculated without knowledge of the ground truth, we show two interesting applications - in the first it is used to guide the selection of difficult samples and in the second to give feedback in a popular Cascaded Pose Regression method for face alignment.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Deep Face Recognition: A Survey (2018, methodology): “Common data augmentation methods consist of photometric transformations [75], [22] and geometric transformations, such as oversampling (multiple patches obtained by cropping at different scales) [22], mirroring [153], and rotating [154] the images.”
Do We Really Need to Collect Millions of Faces for Effective Face Recognition? (2016, methodology): “Popular augmentation methods include simple, geometric transformations such as oversampling (multiple, translated versions of the input image obtained by cropping at different offsets) [16,18], mirroring (horizontal flipping) [3,42], rotating [39] the images as well as various photometric transforma”