TWEETQA: A Social Media Focused Question Answering Dataset

By Wenhan Xiong, Jiawei Wu, Hong Wang, Vivek Kulkarni, Mo Yu, Shiyu Chang, Xiaoxiao Guo, William Yang Wang. 2019

87 citations recorded by the source corpus.

With social media becoming increasingly popular on which lots of news and real-time events are reported, developing automated question answering systems is critical to the effective-ness of many applications that rely on real-time knowledge. While previous datasets have concentrated on question answering (QA) for formal text like news and Wikipedia, we present the first large-scale dataset for QA over social media data. To ensure that the tweets we collected are useful, we only gather tweets used by journalists to write news articles. We then ask human annotators to write questions and answers upon these tweets. Unlike otherQA datasets like SQuAD in which the answers are extractive, we allow the answers to be abstractive. We show that two recently proposed neural models that perform well on formal texts are limited in their performance when applied to our dataset. In addition, even the fine-tuned BERT model is still lagging behind human performance with a large margin. Our results thus point to the need of improved QA systems targeting social media text.

Subfields: Artificial Intelligence, Information Systems, Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yi Tay
2. Mostafa Dehghani
3. Donald Metzler
4. Percy Liang
5. Mohit Bansal
6. Luke S. Zettlemoyer
7. Denny Zhou
8. Quoc V. Le

In Information Systems

1. Yi Tay
2. Wenqiang Lei
3. Tat-seng Chua
4. Dara Bahri
5. Jason Wei
6. Rami Al-Rfou
7. Donald Metzler
8. Soujanya Poria

In Natural Language Processing

1. Yi Tay
2. Donald Metzler
3. Colin Raffel
4. Mostafa Dehghani
5. Mihir Kale
6. Dara Bahri
7. Noah Constant
8. Rami Al-Rfou

In Evaluation & Benchmarks

1. Dara Bahri
2. Jason Wei
3. Rami Al-Rfou
4. Xavier García
5. Tat-seng Chua
6. Yi Tay
7. Adam Roberts
8. Donald Metzler

Citations in the corpus

• ByT5: Towards a Token-Free Future with Pre-trained Byte-to-Byte Models (2021, background): “TweetQA (Xiong et al., 2019) is an abstractive question-answering task built from tweets mentioned in news articles.”
• UL2: Unifying Language Learning Paradigms (2022, methodology): “Concretely, we use RACE (Reading comprehension) (Lai et al., 2017), QASC (Khot et al., 2020), OpenBookQA (Mihaylov et al., 2018), TweetQA (Xiong et al., 2019), QuAIL (Rogers et al., 2020), IMDB (Maas et al., 2011), Agnews (Zhang et al., 2015), DocNLI (Yin et al., 2021), Adversarial NLI (Nie et al.,…”
• CrossFit: A Few-shot Learning Challenge for Cross-task Generalization in NLP (2021, neutral): “…cls/emotion Barbieri et al. 2020 tweet_eval-stance_hillary cls/emotion Barbieri et al. 2020 tweet_qa qa/machine reading comprehension Xiong et al. 2019 web_questions qa/closed-book qa Berant et al. 2013 wiki_auto cls/other Jiang et al. 2020 wiki_bio cg/other Lebret et al. 2016…”
• Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answering (2021, background): “0 Free-form A set of web documents TWEETQA [145] Social media 13.”
• Unifying Language Learning Paradigms (2022, methodology): “Concretely, we use RACE (Reading comprehension) (Lai et al., 2017), QASC (Khot et al., 2020), OpenBookQA (Mihaylov et al., 2018), TweetQA (Xiong et al., 2019), QuAIL (Rogers et al., 2020), IMDB (Maas et al., 2011), Agnews (Zhang et al., 2015), DocNLI (Yin et al., 2021), Adversarial NLI (Nie et al.,…”
• UnifiedQA-v2: Stronger Generalization via Broader Cross-Format Training (2022, unknown): “Small 79.4 80.7 67.6 66.4 77.1 78.0 45.5 53.7 55.9 55.2 50.4 50.0 42.9 47.1 35.9 37.0 80.0 72.8 59.4 60.1 0.7 60 Base 88.2 86.8 78.1 74.7 82.5 82.4 49.8 60.4 70.3 69.0 59.0 59.4 53.0 58.5 42.4 45.7 86.9 86.3 67.8 69.2 1.5 50 Large 91.1 90.8 85.9 86.3 86.1 85.8 51.6 63.2 78.0 78.8 68.2 71.8 65.9 70.5”