How should we evaluate supervised hashing?

By Alexandre Sablayrolles, Matthijs Douze, Nicolas Usunier, Herv'e J'egou. 2016

90 citations recorded by the source corpus.

Hashing produces compact representations for documents, to perform tasks like classification or retrieval based on these short codes. When hashing is supervised, the codes are trained using labels on the training data. This paper first shows that the evaluation protocols used in the literature for supervised hashing are not satisfactory: we show that a trivial solution that encodes the output of a classifier significantly outperforms existing supervised or semi-supervised methods, while using much shorter codes. We then propose two alternative protocols for supervised hashing: one based on retrieval on a disjoint set of classes, and another based on transfer learning to new classes. We provide two baseline methods for image-related tasks to assess the performance of (semi-)supervised hashing: without coding and with unsupervised codes. These baselines give a lower- and upper-bound on the performance of a supervised hashing scheme.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Information Retrieval & Search, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Information Retrieval & Search

In Evaluation & Benchmarks

Citations in the corpus

FastText.zip: Compressing text classification models (2016, methodology): “Section 4 shows the effectiveness of our approach in experiments on multiple text classiﬁcation benchmarks.”
Deep Learning for Free-Hand Sketch: A Survey (2020, background): “In recent years, motivated by the zero-shot setting for supervised photo retrieval [241], zero-shot sketch-based image retrieval (ZSSBIR) has also been studied [65], [216], [242], [243], [244],”