Rethinking the Objectives of Extractive Question Answering
By Martin Fajcik, Josef Jon, Santosh Kesiraju, Pavel Smrz. 2020
12 citations recorded by the source corpus.
This work demonstrates that using the objective with independence assumption for modelling the span probability P (a_s , a_e ) = P (a_s )P (a_e) of span starting at position a_s and ending at position a_e has adverse effects. Therefore we propose multiple approaches to modelling joint probability P (a_s , a_e) directly. Among those, we propose a compound objective, composed from the joint probability while still keeping the objective with independence assumption as an auxiliary objective. We find that the compound objective is consistently superior or equal to other assumptions in exact match. Additionally, we identified common errors caused by the assumption of independence and manually checked the counterpart predictions, demonstrating the impact of the compound objective on the real examples. Our findings are supported via experiments with three extractive QA models (BIDAF, BERT, ALBERT) over six datasets and our code, individual results and manual analysis are available online.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Computer Vision and Pattern Recognition
In Natural Language Processing
Citations in the corpus
- R2-D2: A Modular Baseline for Open-Domain Question Answering (2021, methodology): “…probabilities of: • token s being starting token of an answer span, • token e being ending token of an answer span, • tokens s and e being boundary tokens of an answer span (Fajcik et al., 2020), • passage p containing an answer for the question q (inner reranker) as in Karpukhin et al. (2020).”