Alignment over Heterogeneous Embeddings for Question Answering

By Vikas Yadav, Steven Bethard, Mihai Surdeanu. 2019

32 citations recorded by the source corpus.

We propose a simple, fast, and mostly-unsupervised approach for non-factoid question answering (QA) called Alignment over Heterogeneous Embeddings (AHE). AHE simply aligns each word in the question and candidate answer with the most similar word in the retrieved supporting paragraph, and weighs each alignment score with the inverse document frequency of the corresponding question/answer term. AHE’s similarity function operates over embeddings that model the underlying text at different levels of abstraction: character (FLAIR), word (BERT and GloVe), and sentence (InferSent), where the latter is the only supervised component in the proposed approach. Despite its simplicity and lack of supervision, AHE obtains a new state-of-the-art performance on the “Easy” partition of the AI2 Reasoning Challenge (ARC) dataset (64.6% accuracy), top-two performance on the “Challenge” partition of ARC (34.1%), and top-three performance on the WikiQA dataset (74.08% MRR), outperforming many other complex, supervised approaches. Our error analysis indicates that alignments over character, word, and sentence embeddings capture substantially different semantic information. We exploit this with a simple meta-classifier that learns how much to trust the predictions over each representation, which further improves the performance of unsupervised AHE.

Subfields: Artificial Intelligence, Management Science and Operations Research, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Management Science and Operations Research

In Natural Language Processing

Citations in the corpus

Improving Question Answering with External Knowledge (2019, methodology): “Most previous methods (Khashabi et al., 2017; Musa et al., 2018; Ni et al., 2019; Yadav et al., 2019) perform information retrieval on the reference corpus to retrieve relevant sentences to form reference documents.”
How Context Affects Language Models' Factual Predictions (2020, methodology): “Radford et al. [2019] and Lewis et al. [2019b] demonstrate non-trivial performance on CoQA [Reddy et al., 2019] and SQuAD [Rajpurkar et al., 2016] respectively, and [Yadav et al., 2019] achieve SoTA results using an unsupervised method for multi-choice QA on ARC [Clark et al., 2018].”
Case-Based Abductive Natural Language Inference (2020, methodology): “…an optimal graph, conditioned on a set of semantic constraints (Khashabi et al., 2018; Khot et al., 2017; Jansen et al., 2017; Khashabi et al., 2016), or adopting iterative methods, using sparse or dense encoding mechanisms (Yadav et al., 2019a,b; Pirtoaca et al., 2019; Kundu et al., 2019).”
Quick and (not so) Dirty: Unsupervised Selection of Justification Sentences for Multi-hop Question Answering (2019, methodology): “The third category of methods utilize IR techniques to retrieve justifications from both unstructured (Yadav et al., 2019) and structured (Khashabi et al.”
Explainable Natural Language Reasoning via Conceptual Unification (2020, methodology): “In line with this work, we demonstrate that the use of uniﬁcation patterns for multi-hop explanations can enhance both accuracy and explainability of neural models on a challenging question answering task (Rajani et al., 2019; Yadav et al., 2019b).”