Logical Natural Language Generation from Open-Domain Tables
By Wenhu Chen, Jianshu Chen, Yunde Su, Zhiyu Chen, William Yang Wang. 2020
180 citations recorded by the source corpus.
Neural natural language generation (NLG) models have recently shown remarkable progress in fluency and coherence. However, existing studies on neural NLG are primarily focused on surface-level realizations with limited emphasis on logical inference, an important aspect of human thinking and language. In this paper, we suggest a new NLG task where a model is tasked with generating natural language statements that can be logically entailed by the facts in an open-domain semi-structured table. To facilitate the study of the proposed logical NLG problem, we use the existing TabFact dataset~(CITATION) featured with a wide range of logical/symbolic inferences as our testbed, and propose new automatic metrics to evaluate the fidelity of generation models w.r.t. logical inference. The new task poses challenges to the existing monotonic generation frameworks due to the mismatch between sequence order and logical order. In our experiments, we comprehensively survey different generation architectures (LSTM, Transformer, Pre-Trained LM) trained with different algorithms (RL, Adversarial Training, Coarse-to-Fine) on the dataset and made following observations: 1) Pre-Trained LM can significantly boost both the fluency and logical fidelity metrics, 2) RL and Adversarial Training are trading fluency for fidelity, 3) Coarse-to-Fine generation can help partially alleviate the fidelity issue while maintaining high language fluency. The code and data are available at https://github.com/wenhuchen/LogicNLG.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Natural Language Generation
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Yejin Choi
- Luke S. Zettlemoyer
- Percy Liang
- Caiming Xiong
- Noah A. Smith
- Hannaneh Hajishirzi
- Jianfeng Gao
In Computer Vision and Pattern Recognition
- Mohit Bansal
- Yejin Choi
- Luc Van Gool
- Dacheng Tao
- Jianfeng Gao
- Trevor Darrell
- Caiming Xiong
- Cordelia Schmid
In Natural Language Processing
- Yejin Choi
- Luke S. Zettlemoyer
- Mohit Bansal
- Caiming Xiong
- Percy Liang
- Noah A. Smith
- Hannaneh Hajishirzi
- Graham Neubig
In Natural Language Generation
- Rui Zhang
- William Yang Wang
- Yejin Choi
- Luke S. Zettlemoyer
- Richard Socher
- Caiming Xiong
- Sebastian Gehrmann
- Victor Zhong
Citations in the corpus
- DART: Open-Domain Structured Data Record to Text Generation (2020, background): “…such as slot-value pairs (e.g., WikiBio (Lebret et al., 2016), The E2E Dataset (Novikova et al., 2017b), dialog response generation (Wen et al., 2016)) and tables (e.g., RotoWire (Wiseman et al., 2017), WikiTableText (Bao et al., 2018), LogicNLP (Chen et al., 2020a), ToTTo (Parikh et al., 2020)).”
- UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models (2022, methodology): “• Formal language to text translation describes formal language in natural language (Chen et al., 2020d).”
- ToTTo: A Controlled Table-To-Text Generation Dataset (2020, methodology): “Target Quality Target Source Content Selection Wikibio (Lebret et al., 2016) 583K Biographies Noisy Wikipedia Not specified Rotowire (Wiseman et al., 2017) 4.9K Basketball Noisy Rotowire Not specified WebNLG (Gardent et al., 2017b) 25.3K 15 DBPedia categories Clean Annotator Generated Fully specified E”
- Right for the Right Reason: Evidence Extraction for Trustworthy Tabular Reasoning (2022, unknown): “, 2021, inter alia), and table-to-text generation (e.g., Parikh et al., 2020; Li et al., 2021; Nan et al., 2021; Yoran et al., 2021; Chen et al., 2020a).”
- Logic2Text: High-Fidelity Natural Language Generation from Logical Forms (2020, methodology): “We conduct experiment without using the logical form, i.e., only feed the table caption, table header and the table content as input, which is the task setting of generating arbitrary logically correct descriptions in (Chen et al., 2020).”
- De-Confounded Variational Encoder-Decoder for Logical Table-to-Text Generation (2021, methodology): “We conduct experiments on two datasets: LogicNLG (Chen et al., 2020a) and Logic2Text (Chen et al.”
- Towards Table-to-Text Generation with Numerical Reasoning (2021, methodology): “…sources in natural language tasks, such as table-to-text generation (Liang et al., 2009; Wiseman et al., 2017; Lebret et al., 2016; Parikh et al., 2020), table question answering (Pasupat and Liang, 2015; Wang et al., 2018), and table-based fact verification (Chen et al., 2020b; Gupta et al., 2020).”
- FeTaQA: Free-form Table Question Answering (2021, methodology): “TabFact (Chen et al., 2020d) is composed of Wikipedia tables coupled with statements labeled as either “ENTAILED” or “RE-FUTED” by the table.”
- NeuroLogic A*esque Decoding: Constrained Text Generation with Lookahead Heuristics (2021, methodology): “Then we compare our approach, N EURO L OGIC on top of GPT-2, with previous table-to-text meth-ods, including TGen (Dušek and Jurˇcíˇcek, 2016), Template-GPT-2 (Chen et al., 2020a), KGPT (Chen et al., 2020b), in multiple few-shot settings with various numbers of training instances.”
- ESPRIT: Explaining Solutions to Physical Reasoning Tasks (2020, methodology): “, 2016) have shown promising results on table-to-text tasks (Wiseman et al., 2017; Gehrmann et al., 2018; Puduppully et al., 2019a,b; Iso et al., 2019; Castro Ferreira et al., 2019; Zhao et al., 2020; Chen et al., 2020a).”
- Large Language Models are few(1)-shot Table Reasoners (2022, background): “…2021b; Talmor et al., 2020; Chen et al., 2020a; Nan et al., 2022), table fact verifi-cation (Chen et al., 2019; Aly et al., 2021), table-based generation (Chen et al., 2020b; Parikh et al., 2020; Nan et al., 2021), and table-grounded conversation (Budzianowski et al., 2018; Nakamura et al., 2022).”
- Logic-Consistency Text Generation from Semantic Parses (2021, methodology): “, 2017b), LogicNLP(Chen et al., 2020a), RotoWire(Wiseman et al.”