Transformer-XH: Multi-Evidence Reasoning with eXtra Hop Attention

By Chen Zhao, Chenyan Xiong, Corby Rosset, Xia Song, Paul N. Bennett, Saurabh Tiwary. 2020

125 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

Citations in the corpus

Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval (2020, background): “, BM25, which has become the bottleneck of many systems (Nogueira & Cho, 2019; Luan et al., 2020; Zhao et al., 2020).”
Adaptive Information Seeking for Open-Domain Question Answering (2021, methodology): “For example, some works employ a single retrieval function multiple times (Das et al., 2019a; Qi et al., 2019; Xiong et al., 2021), and the other works use a pre-deﬁned sequence of retrieval functions (Asai et al., 2020; Dhingra et al., 2020).”
Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval (2020, methodology): “GoldEn Retriever (Qi et al., 2019) 37.9 48.6 30.7 64,2 18.9 39.1 Semantic Retrieval (Nie et al., 2019) 46.5 58.8 39.9 71.5 26.6 49.2 Transformer-XH (Zhao et al., 2020) 51.6 64.1 40.9 71.4 26.1 51.3 HGN (Fang et al., 2019) 56.7 69.2 50.0 76.4 35.6 59.9 DrKIT (Dhingra et al., 2020) 42.1 51.7 37.1…”
Hierarchical Graph Network for Multi-hop Question Answering (2019, neutral): “60 Transformer-XH (Zhao et al., 2020) 48.”
Retrieve, Rerank, Read, then Iterate: Answering Open-Domain Questions of Arbitrary Complexity from Text (2020, methodology): “…these retrieve-and-read systems, including making use of hyper-links between Wikipedia articles (Nie et al., 2019; Feldman and El-Yaniv, 2019; Zhao et al., 2019; Asai et al., 2020; Dhingra et al., 2020; Zhao et al., 2019) and iterative retrieval (Talmor and Berant, 2018; Das et al., 2019; Qi…”
Rethinking Search: Making Experts out of Dilettantes (2021, background): “Other forms of inductive biases such as multi-hop reasoning [2, 13, 107] might also be useful.”
DDRQA: Dynamic Document Reranking for Open-domain Multi-hop Question Answering (2020, methodology): “(Feldman and El-Yaniv, 2019) 31.1 40.4 – 30.6 40.2 GoldEn Retriever (Qi et al., 2019) – 49.7 – 37.9 48.5 DrKIT (Dhingra et al., 2020) 37.9 48.6 – 42.1 51.7 Transformer-XH (Zhao et al., 2020) 54.0 7 points improvement on Paragraph EM over previous best model (Asai et al., 2020), demonstrating the eff”
Answer Complex Questions: Path Ranker Is All You Need (2021, methodology): “, the question in Figure 1) need more than one evidence piece to reason, and each evidence piece can be associated through bridge entities [31].”
Long Document Ranking with Query-Directed Sparse Transformer (2020, methodology): “• Transformer-XH (Zhao et al., 2020) retroﬁts data pipelines to create independent sentences which are fed into BERT models, and aggregates them with an extra-hop attention layer.”
Fine-grained Fact Verification with Kernel Graph Attention Network (2019, methodology): “Some work (Zhao et al., 2020; Ye et al., 2020) also employs our BERT based sentence retrieval in their experiments.”
SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarization (2021, background): “Future work could explore a number of improvements: combining the predictions of multiple NLI models, or combining multiple granularitiy levels, for example through multi-hop reasoning (Zhao et al., 2019).”
Answering Any-hop Open-domain Questions with Iterative Document Reranking (2020, methodology): “BERT Reranker [5], DrKIT [8] and Transformer-XH [43] employ the entities in the question and the retrieved documents to link additional documents, which expands the one-shot retrieval results and improves the evidence coverage.”
Baleen: Robust Multi-Hop Reasoning at Scale via Condensed Retrieval (2021, background): “Multi-hop open-domain models To conduct the “hops”, many prior multi-hop systems (e.g., Asai et al. [1]; Zhao et al. [32]) assume explicit links that connect every passage with related entities.”
F1 Is Not Enough! Models and Evaluation towards User-Centered Explainable Question Answering (2020, background): “Other approaches are based on question decomposition (Min et al., 2019; Perez et al., 2020), graph/hierarchical structures (Tu et al., 2019; Fang et al., 2019; Asai et al., 2020), virtual knowledge bases (Dhingra et al., 2020) or transformer models (Zhao et al., 2020).”
LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMs (2024, unknown): “…33.9 GPT-4-Turbo (Achiam et al., 2023) 42.4 Fully-supervised RAG CogQA (Ding et al., 2019) 37.1 DrKIT (Dhingra et al., 2020) 42.1 Transformer-XH (Zhao et al., 2019) 51.6 QAMAT+ (Chen et al., 2023b) 57.6 HGN (Fang et al., 2019) 59.7 PathRetriever (Asai et al., 2019) 60.0 HopRetrieve (Li et al.,…”