Learning a Spelling Error Model from Search Query Logs
By Farooq Ahmad, Grzegorz Kondrak. 2005
127 citations recorded by the source corpus.
Applying the noisy channel model to search query spelling correction requires an error model and a language model. Typically, the error model relies on a weighted string edit distance measure. The weights can be learned from pairs of misspelled words and their corrections. This paper investigates using the Expectation Maximization algorithm to learn edit distance weights directly from search query logs, without relying on a corpus of paired words.
Subfields: Artificial Intelligence, Information Retrieval & Search, Natural Language Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Jianfeng Gao
- Chris Quirk
- Elena Voita
- Ivan Provilkov
- Dmitrii Emelianenko
- L. Venkata Subramaniam
- Fabrizio Silvestri
- Tanveer A. Faruquie
In Information Retrieval & Search
- Ivan Provilkov
- Dmitrii Emelianenko
- L. Venkata Subramaniam
- Fabrizio Silvestri
- Tanveer A. Faruquie
- Jianfeng Gao
- Xiaolong Li
- Shourya Roy
In Natural Language Processing
- Jianfeng Gao
- Chris Quirk
- Ivan Provilkov
- Dmitrii Emelianenko
- L. Venkata Subramaniam
- Fabrizio Silvestri
- Tanveer A. Faruquie
- Xiaolong Li
Citations in the corpus
- BPE-Dropout: Simple and Effective Subword Regularization (2019, methodology): “Edit distance is commonly used to model misspellings (Brill and Moore, 2000; Ahmad and Kondrak, 2005; Pinter et al., 2017).”
- A Large Scale Ranker-Based System for Search Query Spelling Correction (2010, background): “While almost all of the spellers mentioned above are based on a pre-defined dictionary (either a lexicon against which the edit distance is computed, or a set of real-word confusion pairs), recent research on query spelling correction focuses on exploiting noisy Web corpora and query logs to infer k”
- A survey of types of text noise and techniques to handle noisy text (2009, methodology): “• Application of EM (Expectation Maximization) algorithm to optimize the probabilities of edit operations over a set of search queries from the query logs with emphasis on single character edit operation [2].”
- Mining Query Logs: Turning Search Usage Data into Knowledge (2010, background): “This important issue, crucial for letting the spell-checking algorithm to work with high precision, is thoroughly studied by Ahmad and Kondrak [8].”
- Semantic Matching in Search (2014, unknown): “, [151, 3]).”