Semantic Text Matching for Long-Form Documents
By Jyun-Yu Jiang, Mingyang Zhang, Cheng Li, Michael Bendersky, Nadav Golbandi, Marc Najork. 2019
102 citations recorded by the source corpus.
Semantic text matching is one of the most important research problems in many domains, including, but not limited to, information retrieval, question answering, and recommendation. Among the different types of semantic text matching, long-document-to-long-document text matching has many applications, but has rarely been studied. Most existing approaches for semantic text matching have limited success in this setting, due to their inability to capture and distill the main ideas and topics from long-form text. In this paper, we propose a novel Siamese multi-depth attention-based hierarchical recurrent neural network (SMASH RNN) that learns the long-form semantics, and enables long-form document based semantic text matching. In addition to word information, SMASH RNN is using the document structure to improve the representation of long-form documents. Specifically, SMASH RNN synthesizes information from different document structure levels, including paragraphs, sentences, and words. An attention-based hierarchical RNN derives a representation for each document structure level. Then, the representations learned from the different levels are aggregated to learn a more comprehensive semantic representation of the entire document. For semantic text matching, a Siamese structure couples the representations of a pair of documents, and infers a probabilistic score as their similarity. We conduct an extensive empirical evaluation of SMASH RNN with three practical applications, including email attachment suggestion, related article recommendation, and citation recommendation. Experimental results on public data sets demonstrate that SMASH RNN significantly outperforms competitive baseline methods across various classification and ranking scenarios in the context of semantic matching of long-form documents.
Subfields: Artificial Intelligence
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Yi Tay
- Zhiyuan Liu
- Maosong Sun
- Donald Metzler
- Mostafa Dehghani
- Chenyan Xiong
- Dara Bahri
- Jamie Callan
Citations in the corpus
- Long Range Arena: A Benchmark for Efficient Transformers (2020, methodology): “We use the ACL Anthology Network (AAN; Radev et al., 2013) dataset, which identifies if two papers have a citation link, a common setup used in long-form document matching (Jiang et al., 2019; Yang et al., 2020).”
- How Does NLP Benefit Legal System: A Summary of Legal Artificial Intelligence (2020, background): “(3) Semantic matching models in sentence level, ABCNN (Yin et al., 2016), and document level, SMASH-RNN (Jiang et al., 2019).”
- Local Self-Attention over Long Text for Efficient Document Retrieval (2020, unknown): “Several others [8, 14] have highlighted important considerations for designing neural ranking models for documents that are distinct from dealing with passages and other short text.”
- Beyond 512 Tokens: Siamese Multi-depth Transformer-based Hierarchical Encoder for Document Matching (2020, methodology): “We evaluate the proposed model with several benchmark data for long-form text matching [13].”
- Long Document Ranking with Query-Directed Sparse Transformer (2020, background): “The relevance matching between queries and documents often takes place at text segments as opposed to individual tokens (Callan, 1994; Jiang et al., 2019), suggesting that a document term may not need information thousands of words away (Metzler and Croft, 2005; Child et al.”