Learning deep structured semantic models for web search using clickthrough data

By Po-Sen Huang, Xiaodong He, Jianfeng Gao, Li Deng, A. Acero, Larry Heck. 2013

2,275 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Information Retrieval & Search, Large Language Models

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Information Retrieval & Search

In Large Language Models

Citations in the corpus

Dense Passage Retrieval for Open-Domain Question Answering (2020, background): “…pairs to effectively generate a large number of examples, such as the in-batch random negative pairs (Yih et al., 2011; Henderson et al., 2017), or when there exist a big collection of positive and negative pairs (Huang et al., 2013), dense representation can outperform sparse representation.”
Few-shot Learning with Retrieval Augmented Language Models (2022, methodology): “The Contriever uses a dualencoder architecture, where the query and documents are embedded independently by a transformer encoder (Huang et al., 2013; Karpukhin et al., 2020).”
Multi-Task Deep Neural Networks for Natural Language Understanding (2019, background): “The objective for the relevance ranking tasks follows the pairwise learning-to-rank paradigm (Burges et al., 2005; Huang et al., 2013).”
A Hybrid Neural Network Model for Commonsense Reasoning (2019, methodology): “Our results show that HNN, combining the strengths of both DSSM and BERT WSCR , has consistently achieved new state-of-the-art results on all three tasks.”
End-to-End Neural Ad-hoc Ranking with Kernel Pooling (2017, methodology): “For example, DSSM [11] and its convolutional version CDSSM [22] map words to leer-tri-grams, embed query and documents using neural networks built upon the leer-tri-grams, and rank documents using their embedding similarity with the query.”
A Deep Relevance Matching Model for Ad-hoc Retrieval (2016, background): “According to existing literature [12, 17], the core problem in ad-hoc retrieval, i.”
Distilling Knowledge from Reader to Retriever for Question Answering (2020, neutral): “Huang et al. (2013) proposed a deep bag-of-words model, where queries and documents were embedded independently, a technique known as bi-encoder.”
A Corpus and Cloze Evaluation for Deeper Understanding of Commonsense Stories (2016, background): “This model (Huang et al., 2013) sion of the hidden layer is 1000, and the dimension of the embedding vector is 300.”
MS MARCO: A Human Generated MAchine Reading COmprehension Dataset (2016, methodology): “This is a variant of [20] where we use LSTM [19]”
ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT (2020, background): “On the le, Figure 2 (a) illustrates representation-focused rankers, which independently compute an embedding for q and another for d and estimate relevance as a single similarity score between two vectors [13, 41].”
Revealing the Importance of Semantic Retrieval for Machine Reading at Scale (2019, background): “Information Retrieval Success in deep neural networks inspires their application to information retrieval (IR) tasks (Huang et al., 2013; Guo et al., 2016; Mitra et al., 2017; Dehghani et al., 2017).”
Multi-step Retriever-Reader Interaction for Scalable Open-domain Question Answering (2019, methodology): “Lastly, we note that there has been work (Huang et al., 2013) to use deep neural network based models for better retrieval or web search but these model do not train a retriever and a QA model jointly.”
Efficient Natural Language Response Suggestion for Smart Reply (2017, background): “This is similar to Deep Structured Semantic Models, which use feedforward networks to project queries and documents into a common space where the relevance of a document given a query is computed as the cosine distance between them [9].”
Unsupervised Dense Information Retrieval with Contrastive Learning (2021, methodology): “Instead, we propose to use a bi-encoder architecture, where documents and queries are encoded independently (Huang et al., 2013; Karpukhin et al., 2020).”
Pretrained Transformers for Text Ranking: BERT and Beyond (2020, background): “At a high level, there are three current approaches: apply existing transformer models with minimal modifications, adapt existing transformer models, perhaps adding additional architectural elements, and redesign transformer-based architectures from scratch.”
Object-Centric Image Generation from Layouts (2020, methodology): “There has been strong interest in image and caption similarity modules for retrieval (Fang et al. 2015; Huang et al. 2013) and for text-to-image generation, most recently with the DAMSM model proposed in (Xu et al.”
AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks (2017, methodology): “Following [4, 9], we define the loss function as the negative log posterior probability that the images are matched with their corresponding text descriptions (ground truth), i.”