UKP-Athene: Multi-Sentence Textual Entailment for Claim Verification

By Andreas Hanselowski, H. Zhang, Zile Li, Daniil Sorokin, Benjamin Schiller, Claudia Schulz, Iryna Gurevych. 2018

206 citations recorded by the source corpus.

The Fact Extraction and VERification (FEVER) shared task was launched to support the development of systems able to verify claims by extracting supporting or refuting facts from raw text. The shared task organizers provide a large-scale dataset for the consecutive steps involved in claim verification, in particular, document retrieval, fact extraction, and claim classification. In this paper, we present our claim verification pipeline approach, which, according to the preliminary results, scored third in the shared task, out of 23 competing systems. For the document retrieval, we implemented a new entity linking approach. In order to be able to rank candidate facts and classify a claim on the basis of several selected facts, we introduce two extensions to the Enhanced LSTM (ESIM).

Subfields: Artificial Intelligence, Information Systems, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Natural Language Processing

Citations in the corpus

How Decoding Strategies Affect the Verifiability of Generated Text (2019, background): “This solution takes inspiration from Hanselowski et al. (2018) and consists of three main stages: (i) identify relevant Wikipedia pages, as in Hanselowski et al. (2018); (ii) retrieve relevant sentences from such pages; (iii) recognize textual entailment between input and retrieved text.”
Reasoning Over Semantic-Level Graph for Fact Checking (2019, methodology): “• Hanselowski et al. (2018) encode each claim-evidence pair separately, and use a pooling function to aggregate features for prediction.”
Revealing the Importance of Semantic Retrieval for Machine Reading at Scale (2019, background): “The release of FEVER (Thorne et al., 2018) stimulates many recent developments (Nie et al., 2019; Yoneda et al., 2018; Hanselowski et al., 2018) on data-driven neural networks for automatic fact checking.”
The FEVER2.0 Shared Task (2019, methodology): “Team DOMLIN (Stammbach and Neumann, 2019) used the document retrieval module of Hanselowski et al. (2018) and a BERT model for two-staged sentence selection based on the work by (Nie et al., 2019).”
TabFact: A Large-scale Dataset for Table-based Fact Verification (2019, background): “…such as recognizing textual entailment (RTE) (Dagan et al., 2005), natural language inference (NLI) (Bowman et al., 2015), claim veriﬁcation (Popat et al., 2017; Hanselowski et al., 2018; Thorne et al., 2018), and common-sense reasoning (Chierchia & McConnell-Ginet, 2000; Zellers et al., 2018).”
Combining Fact Extraction and Verification with Neural Semantic Matching Networks (2018, methodology): “There are other proposed methods during the FEVER Shared Task in which our system, Yoneda et al. (2018) and Hanselowski et al. (2018) are the top three on the leaderboard.”
Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval (2020, methodology): “Unlike other multi-hop QA datasets (Zhang et al., 2018; Talmor & Berant, 2018; Welbl et al., 2018), where the information sources of the answers are knowledge bases, HotpotQA uses documents in Wikipedia.”
Get Your Vitamin C! Robust Fact Verification with Contrastive Evidence (2021, methodology): “The FEVER dataset (Thorne et al., 2018) fueled the development of many fact-checking models (e.g., see Hanselowski et al., 2018; Nie et al., 2019a,b; Yoneda et al., 2018, inter alia ).”
CLIMATE-FEVER: A Dataset for Verification of Real-World Climate Claims (2020, methodology): “We adopt a pipeline approach for our evidence retrieval and claim validation system similar to the baseline system proposed by fever [1] and similar to virtually all competing implementations that followed [7, 8, 9, 10, 11].”
Kernel Graph Attention Network for Fact Verification (2019, methodology): “Athene (Hanselowski et al., 2018) and UNC NLP (Nie et al., 2018) utilize ESIM to encode claim evidence pairs.”
Generating Fact Checking Explanations (2020, methodology): “, 2018), supervised learning (Hanselowski et al., 2018; Chakrabarty et al., 2018) or sentence similarity scoring (Ma et al.”
Adversarial attacks against Fact Extraction and VERification (2019, methodology): “The Athene system had a high error rate for the entailment preserving transformations: even though the model had an evidence recall of 86.37% for the adversarial instances, the FEVER score reduced by 30.45%.”
WikiFactFind: Semi-automated fact-checking based on Wikipedia (2022, methodology): “Athene team use entity linking and WikiMedia search API for article search [4].”
Fine-grained Fact Verification with Kernel Graph Attention Network (2019, methodology): “The document retrieval step retrieves related Wikipedia pages and is kept the same with previous work (Hanselowski et al., 2018; Zhou et al., 2019; Soleimani et al., 2019).”
Explainable Automated Fact-Checking: A Survey (2020, methodology): “Among these tasks are the following: identifying the severity or nature of false claims, i.e., distinguishing between hoaxes, rumors and satire (Rubin et al., 2016; Rashkin et al., 2017); detecting check-worthy claims (Hassan et al., 2015; Hassan et al., 2017); distinguishing between hyper-partisan ”
DTCA: Decision Tree-based Co-Attention Networks for Explainable Claim Verification (2020, background): “…designs different deep models to exploit semantic matching (Nie et al., 2019; Zhou et al., 2019), semantic conﬂicts (Baly et al., 2018b; Dvo ˇ r ´ ak and Woltran, 2019; Wu and Rao, 2020), and semantic entailments (Hanselowski et al., 2018; Ma et al., 2019) between claims and relevant articles.”
Evaluating adversarial attacks against multiple fact verification systems (2019, methodology): “The models we evaluate were the Neuro-Semantic Matching Network (NSMN) (Nie et al., 2019), HexaF (Yoneda et al., 2018) Enhanced ESIM (Hanselowski et al., 2018) and the Transformer Model (Malon, 2018) as trained by the authors.”
A Multi-Level Attention Model for Evidence-Based Fact Checking (2021, methodology): “First, we required that all the models use the same sentence-selection model, which is Hanselowski et al. (2018)’s ESIM.”