TaPas: Weakly Supervised Table Parsing via Pre-training

By Jonathan Herzig, Pawel Krzysztof Nowak, Thomas Müller, Francesco Piccinno, Julian Martin Eisenschlos. 2020

890 citations recorded by the source corpus.

Answering natural language questions over tables is usually seen as a semantic parsing task. To alleviate the collection cost of full logical forms, one popular approach focuses on weak supervision consisting of denotations instead of logical forms. However, training semantic parsers from weak supervision poses difficulties, and in addition, the generated logical forms are only used as an intermediate step prior to retrieving the denotation. In this paper, we present TaPas, an approach to question answering over tables without generating logical forms. TaPas trains from weak supervision, and predicts the denotation by selecting table cells and optionally applying a corresponding aggregation operator to such selection. TaPas extends BERT’s architecture to encode tables as input, initializes from an effective joint pre-training of text segments and tables crawled from Wikipedia, and is trained end-to-end. We experiment with three different semantic parsing datasets, and find that TaPas outperforms or rivals semantic parsing models by improving state-of-the-art accuracy on SQA from 55.1 to 67.2 and performing on par with the state-of-the-art on WikiSQL and WikiTQ, but with a simpler model architecture. We additionally find that transfer learning, which is trivial in our setting, from WikiSQL to WikiTQ, yields 48.7 accuracy, 4.2 points above the state-of-the-art.

Subfields: Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

Reasoning Like Program Executors (2022, methodology): “(iii) On TAT-QA , we adopt the official baselines, including T A P AS (Herzig et al., 2020), NumNet+ V2 and the SOTA model T AG O P (Zhu et al., 2021).”
A Survey of Data Augmentation Approaches for NLP (2021, background): “Studies have shown that DA becomes less beneﬁcial when applied to out-of-domain data, likely because the distribution of augmented data can substantially differ from the original data (Zhang et al., 2019a; Herzig et al., 2020; Campagna et al., 2020; Zhong et al., 2020).”
UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models (2022, methodology): “Structured knowledge encoding methods have been explored in many prior studies (Bogin et al., 2019; Lin et al., 2019; Yin et al., 2020a; Herzig et al., 2020; Agarwal et al., 2020; Saxena et al., 2020; Yasunaga and Liang, 2020; Yasunaga et al., 2021; Oguz et al., 2021).”
Turning Tables: Generating Examples from Semi-structured Tables for Endowing Language Models with Reasoning Skills (2021, neutral): “Unlike prior work where semi-structured data was used for reasoning over tables or knowledge-bases (Eisenschlos et al., 2020; Yin et al., 2020; Herzig et al., 2020; Yu et al., 2021), here we harness tables to allow LMs to reason over text directly.”
GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsing (2020, methodology): “Pre-training vs. training data augmentation Many recent work (Zhang et al., 2019b; Herzig et al., 2020b; Campagna et al., 2020; Zhong et al., 2020) in semantic parsing and dialog state tracking show that training models on a combination of the extra synthetic data and original training data does…”
MultiModalQA: Complex Question Answering over Text, Tables and Images (2021, methodology): “Table QA Module Following prior work (Herzig et al., 2020), our table QA module takes as input the question Q and the table T , and selects a subset of the table cells and an aggregation operation to compute the final answer.”
DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correction (2023, background): “There is also work on tabular language models that encode both tables and text such as TaBERT (Yin et al., 2020), TaPas (Herzig et al., 2020), and Grappa (Yu et al., 2020).”
Span-based Semantic Parsing for Compositional Generalization (2020, background): “However, other end-to-end models directly predict the answer from the context without an executor, where the context can be an image (Hudson and Manning, 2018; Perez et al., 2018), a table (Herzig et al., 2020), etc.”
Learning to Imagine: Integrating Counterfactual Thinking in Neural Discrete Reasoning (2022, methodology): “Firstly, we decide to choose the approach of explicitly modeling discrete operations, since existing NDR solutions have demonstrated its superiority (Dua et al., 2019; Ran et al., 2019; Herzig et al., 2020; Zhu et al., 2021).”
Neurosymbolic Programming (2021, methodology): “TAPAS (Herzig et al., 2020) instead follows the Neural-λ pipeline (Section 2.”
ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning (2022, methodology): “This module is initialized with the TaPas weights (Herzig et al., 2020) pre-trained on the WikiTQ dataset (Pasupat and Liang, 2015).”
Deep entity matching with pre-trained language models (2020, background): “For numeric data, a good candidate solution would be a hybrid neural network similar to [20, 62] that combines the numeric features with the textual features.”
TABBIE: Pretrained Representations of Tabular Data (2021, methodology): “More recently, these pretraining methods have been extended to jointly learn representations of tables as well as text (Herzig et al., 2020; Yin et al., 2020), which enables improved modeling of tasks such as question answering over tables.”
Large Language Models are few(1)-shot Table Reasoners (2022, methodology): “In this paper, we mainly consider TAPAS (Herzig et al., 2020), TABERT (Yin et al., 2020), and TAPEX (Liu et al., 2021).”
TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance (2021, methodology): “The tabular QA base-line Tapas for WTQ is trained with only tabular data in TAT-QA, showing very limited capability to process hybrid data, as can be seen from its performance.”
Data-QuestEval: A Referenceless Metric for Data-to-Text Semantic Evaluation (2021, background): “While QG/QA modeling per se is out of the scope of this paper, further improvements could be obtained thanks to advances in multimodal QA, a field particularly prolific (Chen et al., 2020; Herzig et al., 2020; Nan et al., 2021).”
ReasonBERT: Pre-trained to Reason with Distant Supervision (2021, methodology): “transformer TAPAS (Herzig et al., 2020) as the encoder to capture the table structure.”