Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data

By John D. Lafferty, Andrew McCallum, Fernando C Pereira. 2001

15,607 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

Citations in the corpus

Structured Attention Networks (2017, background): “We focus on two classes of structured attention: linear-chain conditional random fields (CRFs) (Lafferty et al., 2001) and first-order graph-based dependency parsers (Eisner, 1996).”
Dissecting Contextual Word Embeddings: Architecture and Representation (2018, background): “(2018) with a character based CNN word representation, two biLSTM layers and a conditional random field (CRF) loss (Lafferty et al., 2001).”
Deep Contextualized Word Representations (2018, methodology): “Following recent state-of-the-art systems (Lample et al., 2016; Peters et al., 2017), the base-line model uses pre-trained word embeddings, a character-based CNN representation, two biLSTM layers and a conditional random ﬁeld (CRF) loss (Lafferty et al., 2001), similar to Collobert et al. (2011).”
Semi-supervised sequence tagging with bidirectional language models (2017, methodology): “Instead of using a LM, Li and McCallum (2005) uses a probabilistic generative model to infer contextsensitive latent variables for each token, which are then used as extra features in a supervised CRF tagger (Lafferty et al., 2001).”
To Tune or Not to Tune? Adapting Pretrained Representations to Diverse Tasks (2019, methodology): “For NER, we use a BiLSTM with a CRF layer (Lafferty et al., 2001; Lample et al., 2016).”
Non-local Neural Networks (2017, background): “Long-range dependencies can be modeled by graphical models such as conditional random fields (CRF) [29, 28].”
Energy-Inspired Models: Learning with Sampler-Induced Distributions (2019, methodology): “Different choices of energy function recover well-known probabilistic models including Markov random ﬁelds [35], (restricted) Boltzmann machines [62, 24, 29], and conditional random ﬁelds [40].”
Adaptive Stochastic Dual Coordinate Ascent for Conditional Random Fields (2017, methodology): “The conditional random field (CRF) model (Lafferty et al., 2001) is a common tool in natural language processing and computer vision for structured prediction. The optimization of this model is notoriously challenging. Schmidt et al. (2015) describes a practical implementation of the stochastic aver”
On Structured Prediction Theory with Calibrated Convex Surrogate Losses (2017, methodology): “…of common surrogate losses include the structured hinge-loss [40, 43] Φ SSVM ( f , y ) := max ˆ y ∈Y f ˆ y + L (ˆ y , y ) − f y , the log loss (maximum likelihood learning) used, e.g., in conditional random ﬁelds [22], Φ log ( f , y ) := − f y + log ˆ y ∈Y exp f ˆ y , and their hybrids [19, 37].”
Minding the Gaps for Block Frank-Wolfe Optimization of Structured SVMs (2016, methodology): “…mapping ψi(y) representing the sufficient statistics for an energy function associated with a graphical model (as for a conditional random field (Lafferty et al., 2001)), then the SSVM objective is implicitly optimizing over the marginal polytope for the graphical model (Wainwright & Jordan,…”
Listen, attend and spell: A neural network for large vocabulary conversational speech recognition (2015, methodology): “ble-length sequence to another variable-length sequence, neural networks have to be combined with other sequential models such as Hidden Markov Models (HMMs) [22] and Conditional Random Fields (CRFs) [23]. A drawback of this combining approach is that the resulting models cannot be easily trained en”
Learning to Map Sentences to Logical Form: Structured Classification with Probabilistic Categorial Grammars (2005, methodology): “In defining PCCGs, we make use of a conditional loglinear model that is similar to the model form in conditional random fields (CRFs) (Lafferty et al., 2001) or loglinear models applied to parsing (Ratnaparkhi, Roukos, & Ward, 1994; Johnson, Geman, Canon, Chi, & Riezler, 1999).”
Conditional Random Fields as Recurrent Neural Networks (2015, background): “Here E(x) is called the energy of the configuration x ∈ LN and Z(I) is the partition function [17].”
Sequence Transduction with Recurrent Neural Networks (2012, methodology): “As a discriminative sequential model the transducer has similarities with ‘chain-graph’ conditional random fields (CRFs) (Lafferty et al., 2001).”
Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks (2015, background): “Thus, the model represents the probability distribution of sequences in the most general form - unlike Conditional Random Fields [8] and other models that assume independence between between outputs at different time steps, given latent variable states.”
Edinburgh Neural Machine Translation Systems for WMT 16 (2016, neutral): “We speculate that this is an instance of the label bias problem (Lafferty et al., 2001).”
The Graph Neural Network Model (2009, neutral): “Those techniques include random fields [42], Bayesian networks [43], statistical relational learning [44], transductive learning [45], and semisupervised approaches for graph processing [46].”