Improving Information Extraction by Acquiring External Evidence with Reinforcement Learning

By Karthik Narasimhan, Adam Yala, Regina Barzilay. 2016

153 citations recorded by the source corpus.

Most successful information extraction systems operate with access to a large collection of documents. In this work, we explore the task of acquiring and incorporating external evidence to improve extraction accuracy in domains where the amount of training data is scarce. This process entails issuing search queries, extraction from new sources and reconciliation of extracted values, which are repeated until sufficient evidence is collected. We approach the problem using a reinforcement learning framework where our model learns to select optimal actions based on contextual information. We employ a deep Q-network, trained to optimize a reward function that reflects extraction accuracy while penalizing extra effort. Our experiments on two databases -- of shooting incidents, and food adulteration cases -- demonstrate that our system significantly outperforms traditional extractors and a competitive meta-classifier baseline.

Subfields: Artificial Intelligence, Information Systems, Management Science and Operations Research, Natural Language Processing, Reinforcement Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Management Science and Operations Research

In Natural Language Processing

In Reinforcement Learning

Citations in the corpus

Latent Retrieval for Weakly Supervised Open Domain Question Answering (2019, background): “Consulting external evidence sources with latent retrieval has also been explored in information extraction (Narasimhan et al., 2016).”
Coarse-to-Fine Question Answering for Long Documents (2016, background): “reference resolution (Clark and Manning, 2016), information extraction (Narasimhan et al., 2016), semantic parsing (Andreas et al.”
WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents (2022, background): “Several papers have explored the use of the web for information extraction [34] and retrieval [1], question answering [57, 25], dialog [45], and training language models on webtext [2].”
Constructing Datasets for Multi-hop Reading Comprehension Across Documents (2017, methodology): “panding the document set available to a QA system, either in the form of web navigation (Nogueira and Cho, 2016), or via query reformulation techniques, which often use neural reinforcement learning (Narasimhan et al., 2016; Nogueira and Cho, 2017; Buck et al., 2018). While related, this work ultima”
Task-Oriented Query Reformulation with Reinforcement Learning (2017, methodology): “Perhaps the closest work to ours is that by Narasimhan et al. (2016), in which a reinforcement learning based approach is used to reformulate queries iteratively.”
Mapping Instructions and Visual Observations to Actions with Reinforcement Learning (2017, methodology): “Reinforcement learning with neural networks has been used for various NLP tasks, including text-based games (Narasimhan et al., 2015; He et al., 2016), information extraction (Narasimhan et al., 2016), co-reference resolution (Clark and Manning, 2016), and dialog (Li et al., 2016).”
FVQA: Fact-Based Visual Question Answering (2016, methodology): “The work in [63] performs queries over unstructured Web articles, while ours performs queries over structured relational KBs.”
Quark: Controllable Text Generation with Reinforced Unlearning (2022, methodology): “Previous works have used RL techniques in a wide range of classical NLP applications, such as named entity recognition [41], semantic parsing [89], dependency parsing [79], constituency parsing [16], part-of-speech tagging [6], and information extraction [48].”
MathDQN: Solving Arithmetic Word Problems via Deep Reinforcement Learning (2018, background): “…is that deep Q-network has witnessed success in solving various problems with big search space such as playing text-based games (Narasimhan et al. 2015), information extraction (Narasimhan et al. 2016), text generation (Guo 2015) and object detection in images (Caicedo and Lazebnik 2015).”
Boosting Search Engines with Interactive Agents (2021, neutral): “Narasimhan et al. (2016) use RL to query the web for information extraction.”
Dialog State Tracking with Reinforced Data Augmentation (2019, methodology): “…Learning in NLP. RL is a general purpose framework for decision making and has been applied in many NLP tasks such as information extraction (Narasimhan et al., 2016), relational reasoning (Xiong et al., 2017), sequence learning (Ranzato et al., 2015; Li et al., 2018; Celikyilmaz et al.,…”
The Gap of Semantic Parsing: A Survey on Automatic Math Word Problem Solvers (2018, background): “The motivation is that deep Q-network has witnessed success in solving various problems with big search space such as playing text-based games [31], information extraction [32], text generation [33] and object detection in images [34].”
Learning how to Active Learn: A Deep Reinforcement Learning Approach (2017, unknown): “For example, recently, DRL has been studied for information extraction problem (Narasimhan et al., 2016).”
Reinforcement Learning for Relation Classification From Noisy Data (2018, methodology): “To address these issues, we propose a new framework which ﬁrst selects correct sentences in the framework of reinforcement learning (Sutton and Barto 1998; Narasimhan, Yala, and Barzilay 2016) and then predicts relations from each sentence in the cleansed data.”
Learning Dynamic Context Augmentation for Global Entity Linking (2019, background): “Reinforcement Learning (RL) algorithms have proven to be able to alleviate such accumulated noises in the decision sequence in many recent works (Narasimhan et al., 2016; Feng et al., 2018).”
Decontextualization: Making Sentences Stand-Alone (2021, methodology): “…we presented for open-domain QA; anaphora resolution is crucial for both summarization and machine translation (Su-sanne et al., 1992); and decontextualizing sentences may help in recovering explicit mentions of entities and relations which can help information extraction (Narasimhan et al., 2016).”
A Comprehensive Survey on Automatic Knowledge Graph Construction (2023, background): “Reinforcement Learning (RL) puts NER models to interact with the environment domain through a behavior agency with a reward policy, such as the Markov decision process (MDP-based) model [132] and Q-network enhanced model [133].”