CLIMATE-FEVER: A Dataset for Verification of Real-World Climate Claims

By Thomas Diggelmann, Jordan L. Boyd-Graber, Jannis Bulian, Massimiliano Ciaramita, Markus Leippold. 2020

284 citations recorded by the source corpus.

We introduce CLIMATE-FEVER, a new publicly available dataset for verification of climate change-related claims. By providing a dataset for the research community, we aim to facilitate and encourage work on improving algorithms for retrieving evidential support for climate-specific claims, addressing the underlying language understanding challenges, and ultimately help alleviate the impact of misinformation on climate change. We adapt the methodology of FEVER [1], the largest dataset of artificially designed claims, to real-life claims collected from the Internet. While during this process, we could rely on the expertise of renowned climate scientists, it turned out to be no easy task. We discuss the surprising, subtle complexity of modeling real-world climate-related claims within the \textsc{fever} framework, which we believe provides a valuable challenge for general natural language understanding. We hope that our work will mark the beginning of a new exciting long-term joint effort by the climate science and AI community.

Subfields: Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

Dense Text Retrieval Based on Pretrained Language Models: A Survey (2022, background): “19 pandemic dataset [45], financial dataset [60], biomedical dataset [61], climate-specific dataset [67] and scientific dataset [68].”
FEVEROUS: Fact Extraction and VERification Over Unstructured and Structured information (2021, methodology): “This annotation strategy was adopted to construct a similar dataset for Danish [Nørregaard and Derczynski, 2021], and adapted for real-world climate change-related claims [Diggelmann et al., 2021].”
Evidence-based Verification for Real World Information Needs (2021, methodology): “…et al., 2018; Thorne et al., 2018; Chen et al., 2019; Augenstein et al., 2019; Hanselowski et al., 2019; Chen et al., 2020; Wadden et al., 2020; Diggelmann et al., 2020; Schuster et al., 2021, inter alia ) that have enabled the development of new models and methods for machine reading and…”
Task-aware Retrieval with Instructions (2022, background): “, 2020), Climate- Fever (Diggelmann et al., 2020), and SciFact (Wadden et al.”
ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction (2021, unknown): “, 2018), Climate-FEVER (CFEVER) (Diggelmann et al., 2020), FEVER (Thorne et al.”
BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models (2021, unknown): “76 Fact Checking Wikipedia Climate-FEVER [13] 3 Binary —- —- 1,535 5,416,593 3.”
Retrieval Capabilities of Large Language Models Scale with Pretraining FLOPs (2025, unknown): “ClimateFEVER Diggelmann et al. (2020): similar in spirit to FEVER, ClimateFEVER is a dataset for verification of climate change-related claims.”