TL;DR: Mining Reddit to Learn Automatic Summarization

By Michael Völske, Martin Potthast, Shahbaz Syed, Benno Stein. 2017

274 citations recorded by the source corpus.

Recent advances in automatic text summarization have used deep neural networks to generate high-quality abstractive summaries, but the performance of these models strongly depends on large amounts of suitable training data. We propose a new method for mining social media for author-provided summaries, taking advantage of the common practice of appending a “TL;DR” to long posts. A case study using a large Reddit crawl yields the Webis-TLDR-17 dataset, complementing existing corpora primarily from the news genre. Our technique is likely applicable to other social media sites and general web crawls.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Direct Preference Optimization: Your Language Model is Secretly a Reward Model (2023, methodology): “Following prior work, we use the Reddit TL;DR summarization dataset [38] along with human preferences gathered by Stiennon et al.”
Fine-Tuning Language Models from Human Preferences (2019, methodology): “We also applied our method to two summarization tasks: the CNN/Daily Mail dataset of Hermann et al. (2015) and the TL;DR dataset of Völske et al. (2017).”
PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization (2019, methodology): “The sub-reddit posts strictly follow the rule of writing a descriptive ”TL;DR” summary and has higher qual- 1https://www.tensorflow.org/datasets/ catalog/overview ity than (Völske et al., 2017) (which used more subreddits) based on our manual inspection.”
Learning to summarize from human feedback (2020, methodology): “When applying our methods on a version of the Reddit TL;DR dataset [63], we train policies via human feedback that produce better summaries than much larger policies trained via supervised learning.”
Training Language Models with Language Feedback (2022, methodology): “We follow prior work on learning from human preferences (Stiennon et al., 2020) and learn to summarize Reddit posts from Völske et al. (2017).”
The Pushshift Reddit Dataset (2020, neutral): “Social media data collected by Pushshift has been used already by researchers in computational linguistics and natural language processing [51, 54, 70, 78, 122, 129, 131], recommender systems [21, 38, 66, 69], intelligent conversational systems [1, 59, 80], automatic summarization [120], entity reco”
Understanding the Effects of RLHF on LLM Generalisation and Diversity (2023, methodology): “One notable exception is Stiennon et al. (2022), who perform some experiments evaluating their models trained the TL;DR dataset (Völske et al., 2017) (reddit post summarisation) on the CNN/Daily Mail dataset (Nallapati et al., 2016) (news article summarisation).”
Privacy Regularization: Joint Privacy-Utility Optimization in LanguageModels (2021, unknown): “The privacy loss becomes: 3 Evaluation In our experiments, we use a subset of the Avo-cado corporate email dataset (Oard et al., 2015) with 100 users and 60,000 samples and a subset of Reddit dataset (Völske et al., 2017) with 10,000 users and 3 million samples.”
Training Language Models with Language Feedback at Scale (2023, methodology): “The human summaries we collect are of excellent quality, as demonstrated in a human evaluation, where we compare our human-written summaries to the ones automatically extracted from Reddit (Völske et al., 2017) (also used as baselines in Stiennon et al.”
Statistical Rejection Sampling Improves Preference Optimization (2023, methodology): “D tldrsft is a filtered version of Reddit TL;DR dataset (V¨olske et al., 2017).”
Training Language Models with Natural Language Feedback (2022, methodology): “We follow prior work on learning from human preferences (Stiennon et al., 2020) and learn to summarize Reddit posts from Völske et al. (2017).”
Intrinsic Evaluation of Summarization Datasets (2020, background): “TL;DR (Völske et al., 2017) is a dataset of userwritten articles from the social media platform Reddit along with the author-provided courtesy summaries that tend to be multi-sentence.”
Generating Abstractive Summaries with Finetuned Language Models (2019, methodology): “To avoid this problem, Kim et al. (2018) and V¨olske et al. (2017) created reddit-based corpora with more abstractive target summaries that enable the evaluation of supervised models instead.”
Learning from Natural Language Feedback (2024, methodology): “We follow prior work on learning from human preferences (Stiennon et al., 2020) and learn to summarize Reddit posts from Völske et al. (2017).”
Active Evaluation: Efficient NLG Evaluation with Few Pairwise Comparisons (2022, methodology): “, 2020) of GPT3-like transformers on the TL;DR dataset (Völske et al., 2017).”