Table Fact Verification with Structure-Aware Transformer

By Hongzhi Zhang, Yingyao Wang, Sirui Wang, Xuezhi Cao, Fuzheng Zhang, Zhongyuan Wang. 2020

75 citations recorded by the source corpus.

Verifying fact on semi-structured evidence like tables requires the ability to encode structural information and perform symbolic reasoning. Pre-trained language models trained on natural language could not be directly applied to encode tables, because simply linearizing tables into sequences will lose the cell alignment information. To better utilize pre-trained transformers for table representation, we propose a Structure-Aware Transformer (SAT), which injects the table structural information into the mask of the self-attention layer. A method to combine symbolic and linguistic reasoning is also explored for this task. Our method outperforms baseline with 4.93% on TabFact, a large scale table verification dataset.

Subfields: Artificial Intelligence, Management Science and Operations Research, Natural Language Processing, Model Architectures, Information Extraction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Percy Liang
2. Luke S. Zettlemoyer
3. Caiming Xiong
4. Graham Neubig
5. Richard Socher
6. Noah A. Smith
7. Mohit Bansal
8. Jonathan Berant

In Management Science and Operations Research

1. Torsten Scholak
2. Rui Zhang
3. Noah A. Smith
4. Michihiro Yasunaga
5. Luke S. Zettlemoyer
6. Caiming Xiong
7. Ziyu Yao
8. Chien-Sheng Wu

In Natural Language Processing

1. Percy Liang
2. Caiming Xiong
3. Graham Neubig
4. Luke S. Zettlemoyer
5. Richard Socher
6. Jonathan Berant
7. Mirella Lapata
8. Noah A. Smith

In Model Architectures

1. Luke S. Zettlemoyer
2. Noah A. Smith
3. Victor Zhong
4. Julian Martin Eisenschlos
5. Rui Zhang
6. Ming Zhong
7. Michihiro Yasunaga
8. Caiming Xiong

In Information Extraction

1. William W. Cohen
2. Sida I. Wang
3. Luke S. Zettlemoyer
4. Torsten Scholak
5. Noah A. Smith
6. Caiming Xiong
7. Victor Zhong
8. Pengcheng Yin

Citations in the corpus

• UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models (2022, background): “Recently, cell-column alignment is modeled by manipulating the attention matrix of transformers (Zhang et al., 2020; Eisenschlos et al., 2021).”
• MATE: Multi-view Attention for Table Transformer Efficiency (2021, methodology): “Recently, Zhang et al. (2020) proposed SAT, which uses an attention mask to restrict attention to tokens in the same row and same column.”