Towards Faithful Neural Table-to-Text Generation with Content-Matching Constraints

By Zhenyi Wang, Xiaoyang Wang, Bang An, Dong Yu, Changyou Chen. 2020

90 citations recorded by the source corpus.

Text generation from a knowledge base aims to translate knowledge triples to natural language descriptions. Most existing methods ignore the faithfulness between a generated text description and the original table, leading to generated information that goes beyond the content of the table. In this paper, for the first time, we propose a novel Transformer-based generation framework to achieve the goal. The core techniques in our method to enforce faithfulness include a new table-text optimal-transport matching loss and a table-text embedding similarity loss based on the Transformer model. Furthermore, to evaluate faithfulness, we propose a new automatic metric specialized to the table-to-text generation problem. We also provide detailed analysis on each component of our model in our experiments. Automatic and human evaluations show that our framework can significantly outperform state-of-the-art by a large margin.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

Survey of Hallucination in Natural Language Generation (2022, methodology): “[199] modify PARENT and denote this table-focused version as PARENT-T.”
DART: Open-Domain Structured Data Record to Text Generation (2020, background): “Recently, neural encoder-decoder models based on attention and copy mechanisms have shown promising results (Wiseman et al., 2017; Liu et al., 2018; Gehrmann et al., 2018; Puduppully et al., 2018, 2019; Iso et al., 2019; Castro Ferreira et al., 2019; Shen et al., 2019; Roberti et al., 2019; Zhu et a”
Logic-Consistency Text Generation from Semantic Parses (2021, methodology): “The concentration of our work differs from the related highfidelity text generation work (Chen et al., 2020b; Chan et al., 2019; Nie et al., 2018; Tian et al., 2019; Wang et al., 2020a), by attempting to present the panorama of the challenges of logic-consistent text generation instead of focusing o”
Don’t Say What You Don’t Know: Improving the Consistency of Abstractive Summarization by Constraining Beam Search (2022, background): “Some have focused on the data-to-text setting, which presupposes structured input (Tian et al., 2019; Wang et al., 2020b).”
Large Language Models: A Survey (2024, unknown): “• Advanced metrics such as PARENT [149], PARENT-T [150], and Knowledge F1 [151] are utilized when structured knowledge sources are available.”
Towards Faithfulness in Open Domain Table-to-text Generation from an Entity-centric View (2021, methodology): “Note that our implemented transformer has similar model capacity with Wang et al. (2020b), i.e. the same hidden dimension setting.”
Disentangling the Properties of Human Evaluation Methods: A Classification System to Support Comparability, Meta-Evaluation and Reproducibility Testing (2020, background): “In some cases where criterion names are different, it is slightly more evident that criteria are in fact closely related, as with Wang et al. (2020)’s Faithfulness, Cao et al. (2020)’s Content similarity, and Zhou et al. (2020)’s Content preservation, all of which measure the extent to which the…”
Focus Attention: Promoting Faithfulness and Diversity in Summarization (2021, methodology): “Both, Narayan et al. (2018) and Ailem et al. (2019), use a pretrained Latent Dirichlet Allocation (LDA; Blei et al., 2003) model, whereas, Wang et al. (2020c) use Poisson factor analysis (Zhou et al., 2012), to synthesize topic vectors for the input.”
Towards Table-to-Text Generation with Numerical Reasoning (2021, result): “, w|PT |, similar to that used in many prior work (Wang et al., 2020; Chen et al., 2020a; Kale and Rastogi, 2020b), where wi denotes the i-th word in paragraph PT with length |PT |.”
A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generation (2021, methodology): “, 2019) or optimal transport (Wang et al., 2020b), and template/scaffold guided schema with explicit plans (Ma et al.”
A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions (2023, unknown): “When treating the source content as the reference, traditional n-gram overlap-based evaluation metrics, such as ROUGE (Lin, 2004) and PARENT-T (Wang et al., 2020b), can also be applied to assess faithfulness.”
Sketch and Refine: Towards Faithful and Informative Table-to-Text Generation (2021, methodology): “, 2018), a seq2seq model using a dual attention mechanism to consider both key and value information; (4) OptimTrans (Wang et al., 2020), a Transformer based model that incorporates optimal transport matching loss and embedding similarity loss.”
De-Confounded Variational Encoder-Decoder for Logical Table-to-Text Generation (2021, background): “, 2020), and table-to-text (Wang et al., 2020; Parikh et al., 2020) generation.”
CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing (2023, background): “[46] Zhenyi Wang, Xiaoyang Wang, Bang An, Dong Yu, and Changyou Chen.”