Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification
By Peng Zhou, Jun Tian, Zhenyu Qi, B. Li, Hongwei Hao, Bo Xu. 2016
1,977 citations recorded by the source corpus.
Relation classification is an important semantic processing task in the field of natural language processing (NLP). State-ofthe-art systems still rely on lexical resources such as WordNet or NLP systems like dependency parser and named entity recognizers (NER) to get high-level features. Another challenge is that important information can appear at any position in the sentence. To tackle these problems, we propose Attention-Based Bidirectional Long Short-Term Memory Networks(AttBLSTM) to capture the most important semantic information in a sentence. The experimental results on the SemEval-2010 relation classification task show that our method outperforms most of the existing methods, with only word vectors.
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing, Information Extraction
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Percy Liang
- Mohit Bansal
- Erik Cambria
- Zhiyuan Liu
- Maosong Sun
- Luke S. Zettlemoyer
- Jianfeng Gao
- Caiming Xiong
In Computer Vision and Pattern Recognition
- Dacheng Tao
- Mohit Bansal
- Percy Liang
- Soujanya Poria
- Luc Van Gool
- Erik Cambria
- Trevor Darrell
- Xiaogang Wang
In Natural Language Processing
- Mohit Bansal
- Zhiyuan Liu
- Maosong Sun
- Erik Cambria
- Caiming Xiong
- Percy Liang
- Soujanya Poria
- Richard Socher
In Information Extraction
- Yuan Luo
- Roberto Navigli
- Claudio Delli Bovi
- Alessandro Raganato
- Christopher D. Manning
- Yuhao Zhang
- Wei Dang Lu
- Fei Huang
Citations in the corpus
- Position-aware Attention and Supervised Data Improve Slot Filling (2017, methodology): “, 2015a,b), augmenting RNNs with different components (Xu et al., 2016; Zhou et al., 2016), and combining RNNs and CNNs (Vu et al.”
- Attributing Fair Decisions with Attention Interventions (2021, methodology): “It can be applied to both text and tabular data and is inspired by works in attentionbased models in text-classification (Zhou et al. 2016).”
- Recurrent Neural Networks for Classifying Relations in Clinical Notes (2017, positive): “Our models differ from general domain relation classification models [37,38], in that we do not use syntactic/ semantic resources (compared to Yan et al.”
- Text Classification Improved by Integrating Bidirectional LSTM with Two-dimensional Max Pooling (2016, methodology): “The output of the i th word is shown in the following equation: Here, element-wise sum is used to combine the forward and backward pass outputs.”
- Improved Neural Relation Detection for Knowledge Base Question Answering (2017, background): “Recent research benefits a lot from the advancement of deep learning: from word embeddings (Nguyen and Gr-ishman, 2014; Gormley et al., 2015) to deep networks like CNNs and LSTMs (Zeng et al., 2014; dos Santos et al., 2015; Vu et al., 2016) and attention models (Zhou et al., 2016; Wang et al., 2016).”
- Graph Convolution over Pruned Dependency Trees Improves Relation Extraction (2018, methodology): “Zhou et al. (2016) and Wang et al. (2016) proposed to use attention mechanisms over RNN and CNN architectures for this task.”
- Consistent Representation Learning for Continual Relation Extraction (2022, methodology): “*Corresponding Author (Zhou et al., 2016; Soares et al., 2019a) always assume a fixed set of predefined relations and train on a fixed dataset, which cannot handle the growing relation types in real life well.”
- Latent Relational Metric Learning via Memory-based Attention for Collaborative Ranking (2017, background): “Neural attention mechanisms are popular in the fields of computer vision [17, 28] and NLP [14, 22, 30] and are known to improve performance and interpretability of deep learning models.”
- A Study of Various Text Augmentation Techniques for Relation Classification in Free Text (2019, methodology): “Deep Learning Methods Experiments F1 scores and Train Data Size 100% Training Samples 75% Training Samples F1 score No. of Training Samples F1 score No. of Training Samples CNN Baseline 50.74 15917(-) 49.22 11922(-) Similar Words 50.86 ↑ 31834(+) 49.45 ↑ 23844(+) Synonym 51.68 ↑ 31834(+) 49.28 ↑ 238”
- NERO: A Neural Rule Grounding Framework for Label-Efficient Relation Extraction (2020, methodology): “, attention CNN [32], attention RNN [38, 39], and GCN [7, 37]).”
- Adversarial Training for Relation Extraction (2017, methodology): “Recurrent neural networks (RNN) are another popular choice, and have been used in recent work in the form of recurrent CNNs (Cai et al., 2016) and attention RNNs (Zhou et al., 2016).”
- Lifelong Learning of Hate Speech Classification on Social Media (2021, methodology): “We compare our methods with the following methods: Fine-tuning: The model contains two bidirectional LSTM encoders (Hochreiter and Schmid-huber, 1997; Zhou et al., 2016; Liu et al., 2016) to encode the tweet and the group separately.”
- Attention Guided Graph Convolutional Networks for Relation Extraction (2019, background): “(Zeng et al., 2014; Nguyen and Grish-man, 2015; Wang et al., 2016), recurrent neural networks (Zhou et al., 2016; Zhang et al., 2017) the combination of both (Vu et al., 2016) and transformer (Verga et al., 2018).”
- DNA 5-methylcytosine detection and methylation phasing using PacBio circular consensus sequencing (2023, neutral): “The ccsmeth algorithm for 5mCpG detection Recurrent neural network (RNN) and attention mechanism are widely usedarti fi cialneural networksin naturallanguage processing 39,40 .”
- Neural Sequence Learning Models for Word Sense Disambiguation (2017, methodology): “We follow previous work (Vinyals et al., 2015; Zhou et al., 2016) and compute c as the weighted sum of the hidden state vectors h1, .”