Estimation of statistical translation models based on mutual information for ad hoc information retrieval

By Maryam Karimzadehgan, Chengxiang Zhai. 2010

88 citations recorded by the source corpus.

As a principled approach to capturing semantic relations of words in information retrieval, statistical translation models have been shown to outperform simple document language models which rely on exact matching of words in the query and documents. A main challenge in applying translation models to ad hoc information retrieval is to estimate a translation model without training data. Existing work has relied on training on synthetic queries generated based on a document collection. However, this method is computationally expensive and does not have a good coverage of query words. In this paper, we propose an alternative way to estimate a translation model based on normalized mutual information between words, which is less computationally expensive and has better coverage of query words than the synthetic query method of estimation. We also propose to regularize estimated translation probabilities to ensure sufficient probability mass for self-translation. Experiment results show that the proposed mutual information-based estimation method is not only more efficient, but also more effective than the synthetic query-based method, and it can be combined with pseudo-relevance feedback to further improve retrieval accuracy. The results also show that the proposed regularization strategy is effective and can improve retrieval accuracy for both synthetic query-based estimation and mutual information-based estimation.

Subfields: Artificial Intelligence, Information Systems, Information Retrieval & Search

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Information Retrieval & Search

Citations in the corpus

End-to-End Neural Ad-hoc Ranking with Kernel Pooling (2017, neutral): “e translation matrix can be calculated via mutual information in a corpus [12] or using user clicks [6].”
Embedding-based Query Language Models (2016, methodology): “Their proposed method achieves comparable performance to the mutual information-based translation language models [11].”
Integrating and Evaluating Neural Word Embeddings in Information Retrieval (2015, methodology): “The mutual information between terms w and u is computed as (refer to [16] for details): I(w, u) = ∑”
Generalizing Translation Models in the Probabilistic Relevance Framework (2016, methodology): “However, this not being the focus of this study, we use Dirichlet smoothing [38], as many others have done, successfully, before us [12,34,41].”
Document Retrieval Model Through Semantic Linking (2017, background): “…of words generated from a mixture of latent topics, where a latent topic is a probability distribution over the terms or a cluster of weighted terms, or in other work where the likelihood of translating a document to a query is estimated and is used for the purpose of ranking documents [15, 13].”