A tutorial on hidden Markov models and selected applications in speech recognition

By Lawrence R. Rabiner. 1989

Subfields: Natural Language Processing, Model Architectures

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Model Architectures

Citations in the corpus

An Introduction to Probabilistic Programming (2018, background): “5 which denotes a hidden Markov model (HMM) (Rabiner, 1989) with known initial state, transition, and observation distributions governing 16 sequential observations.”
Stochastic variational inference (2012, background): “We now show how stochastic inference arises by applying stochastic optimization to the natural gradients of the variational objective.”
Shaping Belief States with Generative Environment Models for RL (2019, background): “A key concept in POMDPs is the notion of a belief-state, which can be defined as the sufficient statistics of future states [6, 7, 8].”
Hilbert Space Embeddings of Predictive State Representations (2013, neutral): “If the observations are discrete, then Hidden Markov Models (HMMs) [1] or, in the control setting, Input-Output HMMs (IOHMMs) [2], can be used to represent belief as a discrete distribution over latent states.”
Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Search (2020, methodology): “The iterative procedure is actually one example of widely used Viterbi training [19], which maximizes log likelihood of the most likely hidden alignment.”
Accelerated Profile HMM Searches (2011, methodology): “Another approach is rescaling [5,38].”
Hybrid Deterministic-Stochastic Methods for Data Fitting (2011, methodology): “…a kk i and associated parameters x kk (cid:48) , and deﬁnes the probability of a conﬁguration as The normalizing constant Z i is again set so that the distribution sums to one, and it can be computed using a variant on the forward-backward algorithm used in hidden Markov models [30, § III].”
Incorporating Non-local Information into Information Extraction Systems by Gibbs Sampling (2005, methodology): “…such as HMMs, CMMs, and CRFs, it is standard to use the Viterbi algorithm, a dynamic programming algorithm, to infer the most likely hidden state sequence given the input and the model (see, e.g., Rabiner (1989 One can then sample sequences from the conditional distribution deﬁned by the model.”
Very deep convolutional networks for end-to-end speech recognition (2016, background): “esteps the complicated machinery developed for classical ASR [2, 3, 4, 5, 6]. It is able to do this because it is not restricted by the classical independence assumptions of Hidden Markov Model (HMM) [7] and Connectionist Temporal Classiﬁcation (CTC) [8] models. As a result, a single end-to-end mode”
EESEN: End-to-end speech recognition using deep RNN models and WFST-based decoding (2015, background): “The computation of the backward LSTM layer can be represented similarly.”
Fast and accurate recurrent neural network acoustic models for speech recognition (2015, methodology): “The network can be trained to optimize the total log probability of correct labelings for training data as estimated using the network outputs and forward-backward algorithm [16].”
An Algorithmic Perspective on Imitation Learning (2018, methodology): “HMMs have been used in classical speech recognition [Rabiner, 1989], and motion recognition can be performed in the same manner using HMMs [Inamura et al.”
Online Fast Adaptation and Knowledge Accumulation: a New Approach to Continual Learning (2020, background): “For example, the context variable could be the strategy of an opponent in a game [69, 49, 77, 51], regimes in time-series forecasting [56, 19, 9] or the mood of a user when navigating a content platform in recommender systems [25, 72].”
Bidirectional LSTM-CRF Models for Sequence Tagging (2015, methodology): “(5) The dynamic programming (Rabiner, 1989) can be used efficiently to compute [A]i,j and optimal tag sequences for inference.”
Text-to-Image Generation Grounded by Fine-Grained User Attention (2020, methodology): “To address noise in the traces and alignments, we combine three long-standing methods to automatically refine word-object assignments: tf-idf weighting [24], IBM Model 1 [3], and Hidden Markov Models [29].”
Kernel Methods for Relation Extraction (2002, methodology): “Several probabilistic frameworks for modeling sequential data have recently been introduced to alleviate for HMM restrictions.”
Privacy for Free: Posterior Sampling and Stochastic Gradient Monte Carlo (2015, methodology): “In the past few decades, the Bayesian approach has been intensively used in modelling speeches (Rabiner, 1989), text documents (Blei et al., 2003), images/videos (Fei-Fei & Perona, 2005), social networks (Airoldi et al., 2009), brain activity (Penny et al., 2011), and is often considered gold…”
Houdini: Fooling Deep Structured Prediction Models (2017, methodology): “Fortunately, we can use the forward-backward algorithm [30], which is at the core of the CTC, to get the score of a given label y .”