The Pushshift Reddit Dataset

By Jason Baumgartner, Savvas Zannettou, Brian Keegan, Megan Squire, Jeremy Blackburn. 2020

1,099 citations recorded by the source corpus.

Social media data has become crucial to the advancement of scientific understanding. However, even though it has become ubiquitous, just collecting large-scale social media data involves a high degree of engineering skill set and computational resources. In fact, research is often times gated by data engineering problems that must be overcome before analysis can proceed. This has resulted recognition of datasets as meaningful research contributions in and of themselves.Reddit, the so called “front page of the Internet,” in particular has been the subject of numerous scientific studies. Although Reddit is relatively open to data acquisition compared to social media platforms like Facebook and Twitter, the technical barriers to acquisition still remain. Thus, Reddit's millions of subreddits, hundreds of millions of users, and billions of comments are at the same time relatively accessible, but time consuming to collect and analyze systematically.In this paper, we present the Pushshift Reddit dataset. Pushshift is a social media data collection, analysis, and archiving platform that since 2015 has collected Reddit data and made it available to researchers. Pushshift's Reddit dataset is updated in real-time, and includes historical data back to Reddit's inception. In addition to monthly dumps, Pushshift provides computational tools to aid in searching, aggregating, and performing exploratory analysis on the entirety of the dataset. The Pushshift Reddit dataset makes it possible for social media researchers to reduce time spent in the data collection, cleaning, and storage phases of their projects.

Subfields: Computer Networks and Communications, Information Systems, Evaluation & Benchmarks, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Information Systems

In Evaluation & Benchmarks

In Signal Processing

Citations in the corpus

OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization (2022, methodology): “io Reddit (Baumgartner et al., 2020; Roller et al., 2020) using a next-word prediction objective.”
OPT: Open Pre-trained Transformer Language Models (2022, methodology): “…the OPT-175B model was created by a union of ﬁve datasets, including three datasets used by RoBERTa (Liu et al., 2019b), a subset of the Pile (Gao et al., 2021a), along with the Pushshift.io Reddit dataset that was developed in (Baumgartner et al., 2020) and processed in (Roller et al., 2021).”
“I’m sorry to hear that”: Finding New Biases in Language Models with a Holistic Descriptor Dataset (2022, methodology): “We also measure bias in BlenderBot 2.0 (Komeili et al., 2022; Xu et al., 2022), an encoder/decoder model pre-trained on a Reddit dataset extracted by a third party and made available on pushshift.io (Baumgartner et al., 2020).”
Hi, my name is Martha: Using names to measure and mitigate bias in generative dialogue models (2021, methodology): “Most of our bias measurements in this work are performed on BlenderBot (Roller et al., 2020), a generative dialogue model trained on a thirdparty Reddit dump made available on pushshift.io (Baumgartner et al., 2020).”
Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling (2021, methodology): “Much of this focuses on human-produced content on social media platforms, such as Twitter (e.g. Waseem & Hovy, 2016; Wang et al., 2020; Zampieri et al., 2019; 2020), Facebook (Glava ˇ s et al., 2020; Zampieri et al., 2020), or Reddit (Han & Tsvetkov, 2020; Zampieri et al., 2020).”
Recipes for Safety in Open-domain Chatbots (2020, methodology): “io Reddit (Baumgartner et al., 2020) and BST datasets, and then train a simple semisupervised approach, training on both gold and imputed labels, related to the work of Khatri et al.”
Open-Domain Conversational Agents: Current Progress, Open Problems, and Future Directions (2020, methodology): “A second, larger but noisier, dataset where a similar type of setup has been constructed from pushshift.io Reddit has also been built (Mazaré et al., 2018; Baumgartner et al., 2020).”
Retrieval Augmentation Reduces Hallucination in Conversation (2021, background): “, 2020) is a large-scale open-domain dialogue model, pre-trained on dialogue data scraped from social discussions on the web (Baumgartner et al., 2020).”
The Effect of Natural Distribution Shift on Question Answering Models (2020, methodology): “We sampled Reddit posts from the set of all posts across all subreddits during the month of January 2016 in the Pushshift Reddit Corpus (Baumgartner et al., 2020).”
When Life Gives You Lemons, Make Cherryade: Converting Feedback from Bad Responses into Good Labels (2022, methodology): “Existing dialogue models are primarily trained on human-human conversations (Conneau et al., 2019; Baumgartner et al., 2020; Smith et al., 2020).”
Internet-Augmented Dialogue Generation (2021, methodology): “Datasets such as pushshift.io Reddit (Baumgartner et al., 2020), PersonaChat (Zhang et al., 2018) or Empa-thetic Dialogues (Rashkin et al., 2019) (see Huang et al. (2020) for a review) are typically employed to train the weights of a Transformer encoder-decoder.”
Recipes for Building an Open-Domain Chatbot (2020, methodology): “…Humeau et al. (2019), we use a previously existing Reddit dataset extracted and obtained by a third party and made available on pushshift.io (Baumgartner et al., 2020), training to generate a comment conditioned on the full thread leading up to the comment, spanning 1.5B training examples…”
Bot-Adversarial Dialogue for Safe Conversational Agents (2021, methodology): “We can compare these to human numbers, which are actually quite high on pushshift.io Reddit (16.5%), explaining why some of these methods also exhibit safety issues – as they are trained on this data.”
Beyond Goldfish Memory: Long-Term Open-Domain Conversation (2021, methodology): “Recent state-of-the-art open-domain dialogue agents have utilized Daily Dialogue (Li et al., 2017), PersonaChat (Zhang et al., 2018), Empathetic Dialogues (Rashkin et al., 2019), Wizard of Wikipedia (Dinan et al., 2019) and Pushshift.io Reddit (Baumgartner et al., 2020); see Huang et al. (2020) for ”
Hash Layers For Large Sparse Models (2021, background): “io [33], training to generate a comment conditioned on the full thread leading up to the comment, spanning 1.”
Controlling Style in Generated Dialogue (2020, methodology): “…we start from models pre-trained on a previously existing Reddit dataset extracted and obtained by a third party and made available on pushshift.io (Baumgartner et al., 2020) , then finetune our models on four public open-domain conversational datasets from previous work, collectively denoted by…”
SaFeRDialogues: Taking Feedback Gracefully after Conversational Safety Failures (2021, methodology): “We compare our Safety Feedback Recovery fine-tuned models against 4 base models, (1) BST 2.7B, (2) DialoGPT, (3) the pushshift.io Reddit 2.7B model (a 2.7 billion parameter generative dialogue model pretrained using a previously existing Reddit dataset extracted and obtained by a third party that wa”
Concealed Data Poisoning Attacks on NLP Models (2021, methodology): “We follow their data collection pipeline and collect comment data via pushshift.io (Baumgartner et al., 2020).”
DEMix Layers: Disentangling Domains for Modular Language Modeling (2021, background): “io (Baumgartner et al., 2020) 25B (10M) REVIEWS† 30M Amazon product reviews (Ni et al.”