Be Consistent! Improving Procedural Text Comprehension using Label Consistency
By Xinya Du, Bhavana Dalvi, Niket Tandon, Antoine Bosselut, Wen-tau Yih, Peter Clark, Claire Cardie. 2019
31 citations recorded by the source corpus.
Our goal is procedural text comprehension, namely tracking how the properties of entities (e.g., their location) change with time given a procedural text (e.g., a paragraph about photosynthesis, a recipe). This task is challenging as the world is changing throughout the text, and despite recent advances, current systems still struggle with this task. Our approach is to leverage the fact that, for many procedural texts, multiple independent descriptions are readily available, and that predictions from them should be consistent (label consistency). We present a new learning framework that leverages label consistency during training, allowing consistency bias to be built into the model. Evaluation on a standard benchmark dataset for procedural text, ProPara (Dalvi et al., 2018), shows that our approach significantly improves prediction performance (F1) over prior state-of-the-art systems.
Subfields: Artificial Intelligence, Natural Language Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Hinrich Schütze
- Yoav Goldberg
- Mohit Bansal
- Eduard H. Hovy
- Yonatan Belinkov
- Yejin Choi
- Peter Clark
- Yanai Elazar
In Natural Language Processing
- Yoav Goldberg
- Hinrich Schütze
- Mohit Bansal
- Yejin Choi
- Eduard H. Hovy
- Peter Clark
- Yanai Elazar
- Yonatan Belinkov
Citations in the corpus
- Measuring and Improving Consistency in Pretrained Language Models (2021, background): “Work on consistency in other domains includes (Du et al., 2019) where prediction of consistency in procedural text is improved.”
- Logic-Guided Data Augmentation and Regularization for Consistent Question Answering (2020, neutral): “We perform annealing (Kirkpatrick et al., 1983; Li et al., 2019; Du et al., 2019).”
- Enriching a Model's Notion of Belief using a Persistent Memory (2021, background): “PTLMs are known to contain extensive world knowledge (Petroni et al., 2019; Roberts et al., 2020), yet be inconsistent in their answers to probing questions (Ettinger, 2020; Davison et al., 2019; Kassner and Schütze, 2020; Ravichander et al., 2020; Elazar et al., 2021).”