Semantic hashing

By Ruslan Salakhutdinov, Geoffrey E. Hinton. 2009

1,395 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Representation & Self-Supervised Learning

Citations in the corpus

Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation (2013, background): “Binary representations are also useful as keys for a hash table, as in the semantic hashing algo-rithm (Salakhutdinov and Hinton, 2009).”
Representation Learning: A Review and New Perspectives (2012, background): “Another way to get saturated (nearly binary) units is semantic hashing [175].”
The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables (2016, background): “Salakhutdinov & Hinton (2009) binarized real-valued codes of an autoencoder by adding (Gaussian) noise to the logits before passing them through the logistic function.”
Multimodal Deep Learning (2011, background): “While self-taught learning was first motivated with sparse coding, recent work on deep learning [5, 6, 7] have examined how deep sigmoidal networks can be trained to produce useful representations for handwritten digits and text.”
Collaborative Deep Learning for Recommender Systems (2014, background): “On the other hand, deep learning models recently show great potential for learning effective representations and deliver state-of-the-art performance in computer vision [38] and natural language processing [15, 26] applications.”
Deep learning in neural networks: An overview (2014, methodology): “Due to f ’s constant derivative of 1.0, errors backpropagated through a CEC cannot vanish or explode (Section 5.9) but stay as they are (unless they ‘‘flow out’’ of the CEC to other, typically adaptive parts of the NN).”
A practical tutorial on autoencoders for nonlinear feature fusion: Taxonomy, models, software and guidelines (2018, background): “Salakhutdinov and Hinton demonstrated in [132] how to perform what they call semantic hashing through a multi-layer AE.”
Deep Learning: Methods and Applications (2014, methodology): “1 above in this section based on the use of deep autoencoders [165, 314].”
Natural-Parameter Networks: A Class of Probabilistic Neural Networks (2016, background): “Recently neural networks (NN) have achieved state-of-the-art performance in various applications ranging from computer vision [12] to natural language processing [20].”
Estimating or Propagating Gradients Through Stochastic Neurons (2013, methodology): “Dropout noise (Hinton et al. , 2012) and masking noise (in denoising auto-encoders (Vincent et al. , 2008)) is multiplied (just after the neuron non-linearity), while in semantic hashing (Salakhutdinov and Hinton, 2009) noise is added (just before the non-linearity).”
Online and Linear-Time Attention by Enforcing Monotonic Alignments (2017, background): “(12), as was done e.g. in (Frey, 1997; Salakhutdinov & Hinton, 2009; Foerster et al., 2016).”
Ambient Sound Provides Supervision for Visual Learning (2016, methodology): “Binary coding model For the other variation of our model (which we call the Binary model), we use a binary coding scheme [14,31,36] equivalent to a multi-label classification problem.”
Semantic Parsing for Single-Relation Question Answering (2014, background): “In a separate line of research, deep learning based techniques have been proposed for semantic understanding (Mesnil et al., 2013; Huang et al., 2013; Shen et al., 2014b; Salakhutdinov and Hinton, 2009; Tur et al., 2012).”
Bitwise Neural Networks (2016, methodology): “Additionally, even in an environment with the necessary resources being available, speeding up a DNN can greatly improve the user experience when it comes to tasks like searching big databases (Salakhutdinov & Hinton, 2009).”
Fast Decoding in Sequence Models using Discrete Latent Variables (2018, methodology): “Another discretization technique proposed by (Kaiser & Bengio, 2018) that has been recently explored stems from semantic hashing (Salakhutdinov & Hinton, 2009b).”
Learning Discrete Representations via Information Maximizing Self-Augmented Training (2017, methodology): “The pioneering work is semantic hashing, which uses stacked RBM models to learn compact binary representations (Salakhutdinov & Hinton, 2009).”