WikiHow: A Large Scale Text Summarization Dataset

By Mahnaz Koupaee, William Yang Wang. 2018

344 citations recorded by the source corpus.

Sequence-to-sequence models have recently gained the state of the art performance in summarization. However, not too many large-scale high-quality datasets are available and almost all the available ones are mainly news articles with specific writing style. Moreover, abstractive human-style systems involving description of the content at a deeper level require data with higher levels of abstraction. In this paper, we present WikiHow, a dataset of more than 230,000 article and summary pairs extracted and constructed from an online knowledge base written by different human authors. The articles span a wide range of topics and therefore represent high diversity styles. We evaluate the performance of the existing methods on WikiHow to present its challenges and set some baselines to further improve it.

Subfields: Artificial Intelligence, Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization (2020, methodology): “This builds on prior work that collected data from WikiHow for monolingual summarization in English (Koupaee and Wang, 2018).”
PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization (2019, background): “, 2019), instructions (Koupaee and Wang, 2018), emails (Zhang and Tetreault, 2019), patents ar X iv :1 91 2.”
LongForm: Effective Instruction Tuning with Reverse Instructions (2023, neutral): “We now present a comprehensive evaluation of our LongForm models compared to the baselines on LongForm-C test, recipe generation, long form question answering, short story generation, NLU, and ablation of LongForm-C subsets.”
Neural Text Summarization: A Critical Evaluation (2019, background): “Wiki-How (Koupaee and Wang, 2018) is a collection of articles from the WikiHow knowledge base, where each article contains instructions for performing procedural, multi-step tasks covering various areas, including: arts, ﬁnance, travel, and health.”
LFPT5: A Unified Framework for Lifelong Few-shot Language Learning Based on Prompt Tuning of T5 (2021, methodology): “The datasets for summarization include CNNDM containing CNN/DM news (Nallapati et al., 2016), WikiHow containing how-to instructions (Koupaee & Wang, 2018) and Xsum containing BBC news (Narayan et al., 2018).”
GraDA: Graph Generative Data Augmentation for Commonsense Reasoning (2022, neutral): “Explicit reasoning over knowledge graphs has been a popular approach for improving commonsense understanding of QA models. and Wang, 2018) to QA samples for data augmentation.”
Embodied BERT: A Transformer Model for Embodied, Language-guided Visual Task Completion (2021, background): “Task-oriented, instructional language focuses on objects and interactions between objects and actors, as seen in instructional datasets (Damen et al., 2020; Koupaee and Wang, 2018), as a function of the inextricable relationship”
RefSum: Refactoring Neural Summarization (2021, methodology): “WikiHow5 (Koupaee and Wang, 2018) is a largescale dataset constructed from the articles using online WikiHow knowledge base.”
One Embedder, Any Task: Instruction-Finetuned Text Embeddings (2022, neutral): “…30,000 trex (Elsahar et al., 2018) Asymmetric 30,000 pubmed (Sen et al., 2008) Asymmetric 30,000 medmcqa (Pal et al., 2022) Asymmetric 30,000 wikihow (Koupaee and Wang, 2018) Asymmetric 5,000 simple_wiki (Coster and Kauchak, 2011) Asymmetric 5,000 Super-NI (300 datasets) (Wang et al., 2022b)…”
LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression (2024, background): “For Q1, most text compression datasets are ab-stractive (Toutanova et al., 2016; Koupaee and Wang, 2018; Kim et al., 2019), meaning that they treat prompt compression as a generative task where the original prompts are rephrased into condensed ones.”
How Additional Knowledge can Improve Natural Language Commonsense Question Answering (2019, methodology): “We evaluate this using PIQA as the commonsense task and Wik-iHow dataset (Koupaee and Wang, 2018) as the “relevant” external knowledge source.”
AlignScore: Evaluating Factual Consistency with A Unified Alignment Function (2023, neutral): “Summarization WikiHow* (Koupaee and Wang, 2018) binary classification 508 46 157k”
A Systematic Investigation of Commonsense Understanding in Large Language Models (2021, background): “The stories are either video captions taken from AcitivityNet (Heilbron et al., 2015) or WikiHow passages (Koupaee and Wang, 2018).”
FactPEGASUS: Factuality-Aware Pre-training and Fine-tuning for Abstractive Summarization (2022, methodology): “We pre-train on the C4 dataset (Raffel et al., 2020), and evaluate our pre-trained model on three down-stream abstractive summarization datasets: XSum (Narayan et al., 2018), WikiHow (Koupaee and Wang, 2018), and Gigaword (Rush et al., 2015).”
Extractive Summarization as Text Matching (2020, background): “WikiHow (Koupaee and Wang, 2018) is a diverse dataset extracted from an online knowledge base.”
Visuo-Lingustic Question Answering (VLQA) Challenge (2020, background): “…secondary sources, then perform post-processing (if any), then ﬁnally create question-answers that require joint reasoning. such as RecipeQA (Yagcioglu et al., 2018), Wik-iHow (Koupaee and Wang, 2018), PhysicalIQA (Bisk et al., 2019), ART (Bhagavatula et al., 2019) and TQA (Kembhavi et al., 2017).”
CRAFT Your Dataset: Task-Specific Synthetic Dataset Generation Through Corpus Retrieval and Augmentation (2024, positive): “We construct the embedding database from four large-scale corpora: C4 (Raffel et al., 2020), English Wikipedia, Stack Exchange (Flax Sentence Embeddings Team, 2021), and Wiki-How (Koupaee and Wang, 2018).”