SemEval-2019 Task 8: Fact Checking in Community Question Answering Forums

By Tsvetomila Mihaylova, Georgi Karadzhov, Pepa Atanasova, Ramy Baly, Mitra Mohtarami, Preslav Nakov. 2019

63 citations recorded by the source corpus.

We present SemEval-2019 Task 8 on Fact Checking in Community Question Answering Forums, which features two subtasks. Subtask A is about deciding whether a question asks for factual information vs. an opinion/advice vs. just socializing. Subtask B asks to predict whether an answer to a factual question is true, false or not a proper answer. We received 17 official submissions for subtask A and 11 official submissions for Subtask B. For subtask A, all systems improved over the majority class baseline. For Subtask B, all systems were below a majority class baseline, but several systems were very close to it. The leaderboard and the data from the competition can be found at http://competitions.codalab.org/competitions/20022.

Subfields: Artificial Intelligence, Information Systems, Sociology and Political Science, Evaluation & Benchmarks, Information Extraction

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Information Systems

In Sociology and Political Science

In Evaluation & Benchmarks

In Information Extraction

Citations in the corpus

Adapting Language Models for Zero-shot Learning by Meta-tuning on Dataset and Prompt Collections (2021, background): “Are you not concerned by Beatlemania -style hysterical crowds crongregating on you. . . " SemEval 2019 Task 8 (Mihaylova et al., 2019).”
SemEval-2020 Task 11: Detection of Propaganda Techniques in News Articles (2020, methodology): “Other related shared tasks include the FEVER 2018 and 2019 tasks on Fact Extraction and VERiﬁ-cation (Thorne et al., 2018), the SemEval 2017 and 2019 tasks on determining the veracity of rumors (Derczynski et al., 2017; Gorrell et al., 2019) and the SemEval 2019 task on Fact-Checking in Community Qu”
FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation (2023, background): “Most literature assumes a single, atomic claim, some-times modeled with surrounding context (Nakov et al., 2018; Mihaylova et al., 2019; Shaar et al., 2022).”
Meta-tuning Language Models to Answer Prompts Better (2021, background): “Are you not concerned by Beatlemania -style hysterical crowds crongregating on you. . . " SemEval 2019 Task 8 (Mihaylova et al., 2019).”