Modeling Latent-Dynamic in Shallow Parsing: A Latent Conditional Model with Improved Inference

By Xuan Sun, Louis-philippe Morency, Daisuke Okanohara, Yoshimasa Tsuruoka, Junichi Tsujii. 2008

67 citations recorded by the source corpus.

Shallow parsing is one of many NLP tasks that can be reduced to a sequence labeling problem. In this paper we show that the latent-dynamics (i.e., hidden substructure of shallow phrases) constitutes a problem in shallow parsing, and we show that modeling this intermediate structure is useful. By analyzing the automatically learned hidden states, we show how the latent conditional model explicitly learn latent-dynamics. We propose in this paper the Best Label Path (BLP) inference algorithm, which is able to produce the most probable label sequence on latent conditional models. It outperforms two existing inference algorithms. With the BLP inference, the LDCRF model significantly outperforms CRF models on word features, and achieves comparable performance of the most successful shallow parsers on the CoNLL data when further using part-of-speech features.

Subfields: Artificial Intelligence, Information Extraction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Extraction

Citations in the corpus

Natural Language Processing (Almost) from Scratch (2011, background): “Since then, a certain number of systems based on second-order random fields were reported (Sha and Pereira, 2003; McDonald et al., 2005; Sun et al., 2008), all reporting around 94.”
Bidirectional LSTM-CRF Models for Sequence Tagging (2015, background): “For example, It achieves the F1 score of 94.40 for CoNLL2000 chunking, with slight degradation (Sha and Pereira, 2003; Mc-donald et al., 2005; Sun et al., 2008).”
Dependency Tree-based Sentiment Classification using CRFs with Hidden Variables (2010, background): “LatentDynamic Conditional Random Fields (LDCRF) (Morency et al., 2007; Sun et al., 2008) are probabilistic models with hidden variables for sequential labeling, and belief propagation is used for inference.”
Multi-Task Cross-Lingual Sequence Tagging from Scratch (2016, background): “…rely heavily on feature engineering to handcraft language-dependent features, such as character-level morphological features and word-level N-gram patterns (Huang et al., 2015; Toutanova et al., 2003; Sun et al., 2008), making it difﬁcult to share latent representations between different languages.”
Sequence Classification Restricted Boltzmann Machines With Gated Units (2020, methodology): “The results of the experiments as well as the state-of-the-art results reported in [17] and [41]–[44] are shown in Table II.”