Incorporating Non-local Information into Information Extraction Systems by Gibbs Sampling

By Jenny Rose Finkel, Trond Grenager, Christopher D. Manning. 2005

3,464 citations recorded by the source corpus.

Most current statistical natural language processing models use only local features so as to permit dynamic programming in inference, but this makes them unable to fully account for the long distance structure that is prevalent in language use. We show how to solve this dilemma with Gibbs sampling, a simple Monte Carlo method used to perform approximate inference in factored probabilistic models. By using simulated annealing in place of Viterbi decoding in sequence models such as HMMs, CMMs, and CRFs, it is possible to incorporate non-local structure while preserving tractable inference. We use this technique to augment an existing CRF-based information extraction system with long-distance dependency models, enforcing label consistency and extraction template consistency constraints. This technique results in an error reduction of up to 9% over state-of-the-art systems on two established information extraction tasks.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Learning to Solve Arithmetic Word Problems with Verb Categorization (2014, methodology): “A RIS applies the Stanford dependency parser, named entity recognizer and coreference resolution system to the problem text (de Marneffe et al., 2006; Finkel et al., 2005; Raghunathan et al., 2010).”
Semi-Supervised QA with Generative Domain-Adaptive Nets (2017, methodology): “Next, we use Stanford Named Entity Recog-nizer (NER) (Finkel et al., 2005) to assign each word with one of the seven labels, i.e., “Date”, “Money”, “Percent”, “location”, “Organization” and “Time”.”
The Stanford CoreNLP Natural Language Processing Toolkit (2014, methodology): “This is implemented with a discriminative model using a CRF sequence tagger (Finkel et al., 2005).”
Who did What: A Large-Scale Person-Centered Cloze Dataset (2016, methodology): “To convert the question sentence to a cloze question, we first extract named entities using the Stanford NER system (Finkel et al., 2005) and parse the sentence using the Stanford PCFG parser (Klein and Manning, 2003).”
Distant supervision for relation extraction without labeled data (2009, methodology): “We perform named entity tagging using the Stanford four-class named entity tagger (Finkel et al., 2005).”
Semantic Parsing via Paraphrasing (2014, methodology): “We run all questions through the Stanford CoreNLP pipeline (Toutanova and Manning, 2003; Finkel et al., 2005; Klein and Manning, 2003).”
Knowledge Graph Embedding by Translating on Hyperplanes (2014, methodology): “They aligned Freebase relations with the New York Times corpus by tagging entities in text using Stanford NER (Finkel, Grenager, and Manning 2005) and linking them to Freebase IDs through string matching on names.”
Man is to Person as Woman is to Location: Measuring Gender Bias in Named Entity Recognition (2019, methodology): “We used Flair (Akbik, Blythe, and Vollgraf 2018; Akbik et al. 2019), CoreNLP version 3.9 (Manning et al. 2014; Finkel, Grenager, and Manning 2005), and Spacy version 2.1 with small, medium, and large models.2 We test these models against 139 years of U.S. census data3 from years 1880 to 2018.”
Joint Video and Text Parsing for Understanding Events and Answering Queries (2013, methodology): “Existing NER tools such as Stanford NER [55] can be used.”
Bidirectional LSTM-CRF Models for Sequence Tagging (2015, methodology): “These features are similar to the features extracted from Stanford NER tool (Finkel et al., 2005; Wang and Manning, 2013).”
FEQA: A Question Answering Evaluation Framework for Faithfulness Assessment in Abstractive Summarization (2020, methodology): “Speciﬁcally, we mask important text spans in a sentence, including noun phrases extracted by a constituency parser (Kitaev and Klein, 2018) and named entities extracted by the Stanford CoreNLP NER model (Finkel et al., 2005; Manning et al., 2014).”
Embracing data abundance: BookTest Dataset for Reading Comprehension (2016, methodology): “For common noun and named entity detection we use the Stanford POS tagger (Toutanova et al., 2003) and Stanford NER (Finkel et al., 2005) The validation dataset consists of 10 , 000 NE and 10 , 000 CN questions.”
Don’t Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biases (2019, methodology): “The bias-only model uses NER tags, identiﬁed by running the Stanford NER Tag-ger (Finkel et al., 2005) on the passage, as input.”
Assessing The Factual Accuracy of Generated Text (2019, background): “Entities can be identified with named-entity recognition (NER) systems like Chiu and Nichols [5], Finkel et al. [7], Lample et al. [9] that take in 𝑋 and produce the set { 𝑒 } .”
Design Challenges and Misconceptions in Named Entity Recognition (2009, neutral): “The results we obtained on the CoNLL03 test set were consistent with what was reported in (Finkel et al., 2005).”
Towards Understanding Gender Bias in Relation Extraction (2019, background): “of finding all person entities with a Named Entity Recognition system (Finkel et al., 2005) then replacing the names of these entities with corresponding anonymizations.”
Effects of Semantic Features on Machine Learning-Based Drug Name Recognition Systems: Word Embeddings vs. Manually Constructed Dictionaries (2015, background): “CRF is the top choice for DNR since it is one of the most reliable sequence labeling algorithms and has shown good performances on a large number of NER tasks such as NER in the newswire domain [7,8], biomedical NER [9,10] and clinical NER [11,12].”
Effect of Non-linear Deep Architecture in Sequence Labeling (2013, methodology): “We train all models on the standard We used a comprehensive set of features that comes with the standard distribution of Stanford NER model (Finkel et al., 2005).”
ExFaKT: A Framework for Explaining Facts over Knowledge Graphs and Text (2019, methodology): “, [13], is utilized to collect entities from the documents and com-”