Improved Relation Extraction with Feature-Rich Compositional Embedding Models

By Matthew R. Gormley, Mo Yu, Mark Dredze. 2015

181 citations recorded by the source corpus.

Compositional embedding models build a representation (or embedding) for a linguistic structure based on its component word embeddings. We propose a Feature-rich Compositional Embedding Model (FCM) for relation extraction that is expressive, generalizes to new domains, and is easy-to-implement. The key idea is to combine both (unlexicalized) hand-crafted features with learned word embeddings. The model is able to directly tackle the difficulties met by traditional compositional embeddings models, such as handling arbitrary types of sentence annotations and utilizing global information for composition. We test the proposed model on two relation extraction tasks, and demonstrate that our model outperforms both previous compositional models and traditional feature rich models on the ACE 2005 relation extraction task, and the SemEval 2010 relation classification task. The combination of our model and a log-linear classifier with hand-crafted features gives state-of-the-art results.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Zhiyuan Liu
2. Maosong Sun
3. Mohit Bansal
4. Xu Han
5. Wenpeng Yin
6. Xiang Ren
7. Jie Zhou
8. Mo Yu

Citations in the corpus

• A Two-phase Prototypical Network Model for Incremental Few-shot Relation Classification (2020, background): “Currently, conventional deep supervised (Zeng et al., 2014; Gormley et al., 2015) and distantly supervised (Mintz et al.”
• Continual Relation Learning via Episodic Memory Activation and Reconsolidation (2020, methodology): “The conventional RE work, including both supervised RE models (Zelenko et al., 2003; Zhou et al., 2005; Gormley et al., 2015; Socher et al., 2012; Liu et al., 2013; Zeng et al., 2014; Nguyen and Grishman, 2015; dos Santos et al., 2015; Xu et al., 2015; Liu et al., 2015; Miwa and Bansal, 2016) and…”
• FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation (2018, methodology): “Many works have been proposed for this task, including kernel methods (Zelenko et al., 2002; Mooney and Bunescu, 2006), embedding methods (Gormley et al., 2015), and neural methods (Zeng et al., 2014).”
• Improved Neural Relation Detection for Knowledge Base Question Answering (2017, background): “Recent research benefits a lot from the advancement of deep learning: from word embeddings (Nguyen and Grishman, 2014; Gormley et al., 2015) to deep networks like CNNs and LSTMs (Zeng et al.”
• Neural Snowball for Few-Shot Relation Learning (2019, background): “, 2005) and embedding methods (Gormley et al., 2015) to leverage syntactic information to predict relations.”
• Zero-Shot Information Extraction via Chatting with ChatGPT (2023, methodology): “More surprisingly, compared with two supervised models FCM (Gormley et al., 2015) and MultiR (Hoffmann et al., 2011) on NYT11-HRL, ChatIE surpassed them by 2.5% and 5.8% respectively.”
• Approximation-Aware Dependency Parsing by Belief Propagation (2015, background): “Another natural extension of this work is to explore other types of factors: here we considered only log-linear potential functions (commonly used in CRFs), but any differentiable function would be appropriate, such as a neural network (Durrett and Klein, 2015; Gormley et al., 2015b).”
• A Novel Cascade Binary Tagging Framework for Relational Triple Extraction (2019, background): “Early works (Mintz et al., 2009; Gormley et al., 2015) address the task in a pipelined manner.”
• Joint Extraction of Entities and Relations Based on a Novel Tagging Scheme (2017, methodology): “, 2015) is a network embedding method, which is suitable for arbitrary types of information networks; (3) FCM (Gormley et al., 2015) is a compositional model that combines lexicalized linguistic context and word embeddings for relation extraction.”
• Extracting Multiple-Relations in One-Pass with Pre-Trained Transformers (2019, methodology): “These approaches focus either on feature and model architecture selections (Gormley et al., 2015; Nguyen and Gr-ishman, 2015), or on domain adaptation of MRE models (Fu et al., 2017; Shi et al., 2018).”
• Toward Mention Detection Robustness with Recurrent Neural Networks (2016, background): “Recent work has drawn attention to relation extraction (Plank and Moschitti, 2013; Nguyen et al., 2015a; Gormley et al., 2015).”
• Combining Neural Networks and Log-linear Models to Improve Relation Extraction (2015, methodology): “In order to make it compatible with the previous research (Socher et al., 2012; Gormley et al., 2015), for this dataset, besides the word embeddings and the distance embeddings, we apply the name tagging, part of speech tagging and WordNet features (inherited from Socher et al.”
• Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Models (2023, methodology): “These results suggest that on six widely used IE datasets, ChatIE improves performance by an average of 18.98% compared to the original ChatGPT without ChatIE, and outperforms the supervised models FCM and MultiR [21; 30] on the NYT11-HRL dataset.”
• CoType: Joint Extraction of Typed Entities and Relations with Knowledge Bases (2016, methodology): “(2) DS+Kernel [33]: applies bag-of-feature kernel [33] to train a SVM classifier using DL; (3) DS+Logistic [31]: trains a multi-class logistic classifier(4) on DL; (4) DeepWalk [38]: embeds mention-feature co-occurrences and mention-type associations as a homogeneous network (with binary edges); (5)”
• Summary of ChatGPT/GPT-4 Research and Perspective Towards the Future of Large Language Models (2023, methodology): “These results suggest that on six widely used IE datasets, ChatIE improves performance by an average of 18.98% compared to the original ChatGPT without ChatIE, and out-performs the supervised models FCM and MultiR [20, 28] on the NYT11-HRL dataset.”
• More Data, More Relations, More Context and More Openness: A Review and Outlook for Relation Extraction (2020, methodology): “Inspired by the success of embedding models in other NLP tasks (Mikolov et al., 2013a,b), there are also efforts in encoding text into low-dimensional semantic spaces and extracting relations from textual embeddings (Weston et al., 2013; Riedel et al., 2013; Gormley et al., 2015).”
• Hybrid Attention-Based Prototypical Networks for Noisy Few-Shot Relation Classification (2019, methodology): “Conventional algorithms like kernel methods (Zelenko, Aone, and Richardella 2003; Zhou et al. 2005) and embedding methods (Gormley, Yu, and Dredze 2015) are used.”