Relation Classification via Multi-Level Attention CNNs
By Linlin Wang, Zhu Cao, Gerard de Melo, Zhiyuan Liu. 2016
513 citations recorded by the source corpus.
Relation classification is a crucial ingredient in numerous information extraction systems seeking to mine structured facts from text. We propose a novel convolutional neural network architecture for this task, relying on two levels of attention in order to better discern patterns in heterogeneous contexts. This architecture enables endto-end learning from task-specific labeled data, forgoing the need for external knowledge such as explicit dependency structures. Experiments show that our model outperforms previous state-of-the-art methods, including those relying on much richer forms of prior knowledge.
Subfields: Artificial Intelligence, Model Architectures, Information Extraction
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Mohit Bansal
- Zhiyuan Liu
- Maosong Sun
- Luke S. Zettlemoyer
- Percy Liang
- Jianfeng Gao
- Caiming Xiong
- Noah A. Smith
In Model Architectures
In Information Extraction
Citations in the corpus
- Position-aware Attention and Supervised Data Improve Slot Filling (2017, methodology): “This can be explained by noting that the filters in CNNs are essentially a form of “fuzzy n-gram patterns”.”
- Improved Neural Relation Detection for Knowledge Base Question Answering (2017, background): “, 2015) or attention mechanisms (Wang et al., 2016) based on the entity information (e.”
- Graph Convolution over Pruned Dependency Trees Improves Relation Extraction (2018, methodology): “Zhou et al. (2016) and Wang et al. (2016) proposed to use attention mechanisms over RNN and CNN architectures for this task.”
- Reinforcement Learning for Relation Classification From Noisy Data (2018, background): “In (Wang et al. 2016), two levels of attention is proposed in order to better discern patterns in heterogeneous contexts for relation classification.”
- Attention Guided Graph Convolutional Networks for Relation Extraction (2019, background): “Sequence-based models operate only on the word sequences (Zeng et al., 2014; Wang et al., 2016), whereas dependency-based models incorporate dependency trees into the models (Bunescu and Mooney, 2005; Peng et al., 2017).”
- GeoMAN: Multi-level Attention Networks for Geo-sensory Time Series Prediction (2018, background): “Later, researchers developed a number of multilevel attention-based models to select the relevant features and encoder hidden states in different applications [Wang et al., 2016; Yu et al., 2017].”
- Extracting Drug-Drug Interactions with Attention CNNs (2017, methodology): “The attention mechanism extends attention mechanism by Wang et al. (2016) in that it deals with anonymized entities separately from other words and incorporates a smoothing parameter.”
- A Multi-Channel Deep Neural Network for Relation Extraction (2020, methodology): “Another way to capture sentence structure is to combine an attention mechanism with the position embedding [14], [31].”
- A neural joint model for entity and relation extraction from biomedical text (2017, methodology): “For relation classification, there are two state-of-the-art methods using deep neural networks, namely RNNs [24] and convolutional neural networks (CNNs) [25].”
- Drug-Drug Interaction Extraction via Recurrent Neural Network with Multiple Attention Layers (2017, background): “Wang et al. [21] introduce multi-level attention mechanism to CNN in order to emphasize the keywords and ignore the non-critical words during relation detection.”
- Unified Named Entity Recognition as Word-Word Relation Classification (2021, background): “D convolution on the grid, and also show the very prominence on handling relation determination jobs (Zeng et al. 2014; Wang et al. 2016).”
- Enriching Pre-trained Language Model with Entity Information for Relation Classification (2019, background): “Att-Pooling-CNN (Wang et al., 2016) 88.”
- Adversarial Training for Relation Extraction (2017, methodology): “, 2016) and attention CNN (Wang et al., 2016).”
- More Data, More Relations, More Context and More Openness: A Review and Outlook for Relation Extraction (2020, background): “…data, graph neural networks (GNNs) (Zhang et al., 2018; Zhu et al., 2019a) that build word/entity graphs for reasoning, and attention-based neural networks (Zhou et al., 2016; Wang et al., 2016; Xiao and Liu, 2016) that utilize attention mechanism to aggregate global relational information.”
- A Comprehensive Survey on Automatic Knowledge Graph Construction (2023, methodology): “MIML CNN [194] uses a CNN to proceed with each sentence bag, then leverages a cross-sentence pooling operation to derive an entity-pair representation for multi-label relation modeling.”
- Document-Level Relation Extraction with Adaptive Thresholding and Localized Context Pooling (2020, methodology): “Various approaches including sequence-based methods (Zeng et al., 2014; Wang et al., 2016; Zhang et al., 2017), graph-based methods (Miwa and Bansal, 2016; Zhang et al., 2018; Guo et al., 2019; Wu et al., 2019a), transformer-based methods (Alt et al., 2019; Shi and Lin, 2019), and pre-training…”
- NERO: A Neural Rule Grounding Framework for Label-Efficient Relation Extraction (2020, background): “, attention CNN [32], attention RNN [38, 39], and GCN [7, 37]).”