UCL Machine Reading Group: Four Factor Framework For Fact Finding (HexaF)

By Takuma Yoneda, Jeff Mitchell, Johannes Welbl, Pontus Stenetorp, Sebastian Riedel. 2018

101 citations recorded by the source corpus.

In this paper we describe our 2nd place FEVER shared-task system that achieved a FEVER score of 62.52% on the provisional test set (without additional human evaluation), and 65.41% on the development set. Our system is a four stage model consisting of document retrieval, sentence retrieval, natural language inference and aggregation. Retrieval is performed leveraging task-specific features, and then a natural language inference model takes each of the retrieved sentences paired with the claimed fact. The resulting predictions are aggregated across retrieved sentences with a Multi-Layer Perceptron, and re-ranked corresponding to the final prediction.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Avoiding catastrophic forgetting in mitigating model biases in sentence-pair classification with elastic weight consolidation (2020, background): “A common model for fact verification (Pomerleau and Rao, 2017; Thorne et al., 2018) is text-pair classification between a claim and evidence retrieved from a trusted source (Nie et al., 2018; Yoneda et al., 2018; Hanselowski et al., 2018, inter alia).”
Reasoning Over Semantic-Level Graph for Fact Checking (2019, background): “…aggregate pieces of evidence through concatenating evidence sentences into a single string (Nie et al., 2019), classifying each evidence-claim pair separately, merging the results (Yoneda et al., 2018), and encoding each evidence-claim pair followed by pooling operation (Hanselowski et al., 2018).”
Revealing the Importance of Semantic Retrieval for Machine Reading at Scale (2019, background): “The release of FEVER (Thorne et al., 2018) stimulates many recent developments (Nie et al., 2019; Yoneda et al., 2018; Hanselowski et al., 2018) on data-driven neural networks for automatic fact checking.”
Survey of Hallucination in Natural Language Generation (2022, background): “Most of the literature leverages Wikipedia as the knowledge source [97, 182, 221], which is only a small part of world knowledge.”
The FEVER2.0 Shared Task (2019, methodology): “The top 4 submission from the ﬁrst shared task were submitted as baseline systems for this shared task: UNC (Nie et al., 2019), UCLMR (Yoneda et al., 2018), Athene (Hanselowski et al., 2018) and Papelo (Malon, 2018).”
Combining Fact Extraction and Verification with Neural Semantic Matching Networks (2018, methodology): “There are other proposed methods during the FEVER Shared Task in which our system, Yoneda et al. (2018) and Hanselowski et al. (2018) are the top three on the leaderboard.”
Get Your Vitamin C! Robust Fact Verification with Contrastive Evidence (2021, methodology): “The FEVER dataset (Thorne et al., 2018) fueled the development of many fact-checking models (e.g., see Hanselowski et al., 2018; Nie et al., 2019a,b; Yoneda et al., 2018, inter alia ).”
WikiFactFind: Semi-automated fact-checking based on Wikipedia (2022, methodology): “UCL MRG team proposes to use logistic regression for the document and sentences retrieval [18].”
Language Models as Fact Checkers? (2020, background): “Traditional pipeline Traditional fact-checking pipelines (Malon, 2018; Hanselowski et al., 2018; Yoneda et al., 2018; Lee et al., 2018; Nie et al., 2019) access knowledge within an external knowledge base (i.”
Answering while Summarizing: Multi-task Learning for Multi-hop QA with Evidence Extraction (2019, methodology): “In the previous work, the sub tasks are performed using pipelined models (Nie et al., 2019; Yoneda et al., 2018).”
GEAR: Graph-based Evidence Aggregating and Reasoning for Fact Verification (2019, methodology): “Then, Hanselowski et al. (2018); Yoneda et al. (2018); Hidey and Diab (2018) adopt the enhanced sequential inference model (ESIM) (Chen et al., 2017b), a more effective NLI model, to infer the relevance between evidence and claims instead of DAM.”
Fine-grained Fact Verification with Kernel Graph Attention Network (2019, background): “…as shown in Figure 1, a system could first retrieve related evidence sentences from the background corpus, conduct joint reasoning over these sentences, and aggregate the signals to verify the claim integrity (Nie et al., 2019a; Zhou et al., 2019; Yoneda et al., 2018; Hanselowski et al., 2018).”
Adversarial attacks against Fact Extraction and VERification (2019, methodology): “[28] (TF-IDF information retrieval with a decomposable attention model for natural language inference), a modified version of this model with an ESIM+ELMo for the NLI component, and the top four highest performing scoring system from the 2018 task [19, 31, 10, 16].”
Evaluating adversarial attacks against multiple fact verification systems (2019, methodology): “, 2019), HexaF (Yoneda et al., 2018) Enhanced ESIM (Hanselowski et al.”
A Multi-Level Attention Model for Evidence-Based Fact Checking (2021, methodology): “The ﬁrst group represents the top scores of the FEVER shared task, including those of Hanselowski et al. (2018); Yoneda et al. (2018); Nie et al. (2019a).”
Generating Fact Checking Explanations (2020, methodology): “Evidence selection is performed using keyword matching (Malon, 2018; Yoneda et al., 2018), supervised learning (Hanselowski et al.”
Kernel Graph Attention Network for Fact Verification (2019, methodology): “…the background corpus and then leverages NLI techniques, such as transformer and graph neural networks, to conduct joint reasoning over the multiple pieces of retrieved evidence and predict the claim integrity (Nie et al., 2018; Zhou et al., 2019; Yoneda et al., 2018; Hanselowski et al., 2018).”
Program Enhanced Fact Verification with Verbalization and Graph Attention Network (2020, methodology): “Existing work on fact veriﬁcation is mainly based on collecting and using evidences from unstructured text data (Liu et al., 2020; Nie et al., 2019; Hanselowski et al., 2018; Yoneda et al., 2018).”