Normalization Matters in Zero-Shot Learning

By Ivan Skorokhodov, Mohamed Elhoseiny. 2020

5 citations recorded by the source corpus.

An ability to grasp new concepts from their descriptions is one of the key features of human intelligence, and zero-shot learning (ZSL) aims to incorporate this property into machine learning models. In this paper, we theoretically investigate two very popular tricks used in ZSL: "normalize+scale" trick and attributes normalization and show how they help to preserve a signal's variance in a typical model during a forward pass. Next, we demonstrate that these two tricks are not enough to normalize a deep ZSL network. We derive a new initialization scheme, which allows us to demonstrate strong state-of-the-art results on 4 out of 5 commonly used ZSL datasets: SUN, CUB, AwA1, and AwA2 while being on average 2 orders faster than the closest runner-up. Finally, we generalize ZSL to a broader problem -- Continual Zero-Shot Learning (CZSL) and test our ideas in this new setup. The source code to reproduce all the results is available at this https URL.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

1. Chandan Gautam
2. Sethupathy Parameswaran
3. Suresh Sundaram
4. Ashish Mishra
5. Subhankar Ghosh

Citations in the corpus

• Dynamic VAEs with Generative Replay for Continual Zero-shot Learning (2021, methodology): “• AGEM + CZSL[34]: It is an average gradient episodic memory-based CZSL.”
• Generalized Continual Zero-Shot Learning (2020, methodology): “CZSL Setting-1: This setting closely follows the setting used in [34].”