Unifying Question Answering and Text Classification via Span Extraction

By Nitish Shirish Keskar, Bryan McCann, Caiming Xiong, Richard Socher. 2019

21 citations recorded by the source corpus.

Even as pre-trained language encoders such as BERT are shared across many tasks, the output layers of question answering and text classification models are significantly different. Span decoders are frequently used for question answering and fixed-class, classification layers for text classification. We show that this distinction is not necessary, and that both can be unified as span extraction. A unified, span-extraction approach leads to superior or comparable performance in multi-task learning, low-data and supplementary supervised pretraining experiments on several text classification and question answering benchmarks.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

CTRL: A Conditional Transformer Language Model for Controllable Generation (2019, background): “Several works have pointed to natural language as a means for controlling these multi-task systems (McCann et al., 2018; Radford et al., 2019; Keskar et al., 2019), and several point to the beneﬁts of a code book either speciﬁed explicitly (Wu et al., 2016) or learned in a latent space (Kaiser et…”
BAM! Born-Again Multi-Task Networks for Natural Language Understanding (2019, background): “Concurrently with our work, several other recent works also explore ﬁne-tuning BERT us-ing multiple tasks (Phang et al., 2018; Liu et al., 2019b; Keskar et al., 2019).”
mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer (2020, background): “Similar unifying frameworks were proposed by Keskar et al. (2019) and McCann et al. (2018).”
Unifying Vision-and-Language Tasks via Text Generation (2021, background): “Uniﬁed frameworks: One line of work focus on solving natural language processing tasks in a uniﬁed format, such as question answering (Mccann et al., 2018), span prediction (Keskar et al., 2019), or text generation (Raffel et al., 2020; Brown et al., 2020; Khashabi et al., 2020).”
HUBERT Untangles BERT to Improve Transfer across NLP Tasks (2019, methodology): “Transfer learning for transformer-based models has been studied recently: Keskar et al. (2019) and Wang et al. (2019) report improvements in accuracy over BERT after training on an intermediate task from GLUE; an approach which has come to be known as Supplementary Training on Intermediate Labeled…”