Ultra-Fine Entity Typing

By Eunsol Choi, Omer Levy, Yejin Choi, Luke S. Zettlemoyer. 2018

230 citations recorded by the source corpus.

We introduce a new entity typing task: given a sentence with an entity mention, the goal is to predict a set of free-form phrases (e.g. skyscraper, songwriter, or criminal) that describe appropriate types for the target entity. This formulation allows us to use a new type of distant supervision at large scale: head words, which indicate the type of the noun phrases they appear in. We show that these ultra-fine types can be crowd-sourced, and introduce new evaluation sets that are much more diverse and fine-grained than existing benchmarks. We present a model that can predict ultra-fine types, and is trained using a multitask objective that pools our new head-word supervision with prior supervision from entity linking. Experimental results demonstrate that our model is effective in predicting entity types at varying granularity; it achieves state of the art performance on an existing fine-grained entity typing benchmark, and sets baselines for our newly-introduced datasets.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Natural Language Processing

Citations in the corpus

SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems (2019, methodology): “It was challenging to train annotators to do well on Quora Insincere Questions 10 , Empa-thetic Reactions (Buechel et al., 2018), and a recast version of Ultra-Fine Entity Typing (Choi et al., 2018b, see Appendix C.3 for details), leading to low human performance.”
Knowledge Enhanced Contextual Word Representations (2019, methodology): “Entity typing We also evaluated KnowBertW+W using the entity typing dataset from Choi et al. (2018). To directly compare to ERNIE, we adopted the evaluation protocol in Zhang et al.”
ERNIE: Enhanced Language Representation with Informative Entities (2019, methodology): “UFET is proposed with the Open Entity dataset, which uses a Bi-LSTM for context representation instead of two Bi-LSTMs separated by entity mentions in NFGEC.”
K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters (2020, methodology): “For this task, we evaluate our models on Open Entity (Choi et al., 2018) and FIGER (Ling et al., 2015) following the same split setting as Zhang et al. (2019).”
KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation (2019, background): “Pre-training in NLP There has been a long history of pre-training in NLP. Early works focus on distributed word representations (Collobert and Weston, 2008; Mikolov et al., 2013; Pennington et al., 2014), many of which are often adopted in current models as word embeddings.”
Learning to Few-Shot Learn Across Diverse Natural Language Classification Tasks (2019, methodology): “Entity Typing: We use the following datasets for entity typing: CoNLL-2003 (Sang and De Meulder, 2003), MIT-Restaurant (Liu et al., 2013), and OpenEntity (Choi et al., 2018).”
Few-NERD: A Few-shot Named Entity Recognition Dataset (2021, methodology): “3200 entity annotation, which is inspired and modified from previous fine-grained entity recognition studies (Ling and Weld, 2012; Gillick et al., 2014; Choi et al., 2018; Ringland et al., 2019).”
Learning Cross-Context Entity Representations from Text (2020, background): “Mention-level entity typing Another well studied task is mention-level entity typing (e.g. Ling & Weld, 2012; Choi et al., 2018).”
Fine-Grained Entity Typing for Domain Independent Entity Linking (2019, methodology): “Our typing model itself is adapted from past work on ultra-ﬁne grained entity typing in a different setting (Choi et al., 2018; Onoe and Durrett, 2019).”
CoLAKE: Contextualized Language and Knowledge Embedding (2020, unknown): “We evaluate CoLAKE on Open Entity (Choi et al., 2018).”
Improving Entity Linking by Modeling Latent Entity Type Information (2020, methodology): “13 As shown in Table 5, the ultra-fine entity typing system (Choi et al. 2018) only achieves 26.”
Asking without Telling: Exploring Latent Ontologies in Contextual Representations (2020, background): “…et al., 1993) or Universal Depen-dencies (Nivre et al., 2015), semantic roles from PropBank (Palmer et al., 2005) or Dowty (1991)’s Proto-Roles (Reisinger et al., 2015), and named entities, which have a variety of available ontologies (Pradhan et al., 2007; Ling and Weld, 2012; Choi et al., 2018).”
Probing Toxic Content in Large Pre-Trained Language Models (2021, methodology): “PTLMs have even been used as label generation components in tasks such as entity type prediction (Choi et al., 2018).”
LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention (2020, methodology): “…baseline models, including RoBERTa, in all experiments, and obtains state-of-the-art results on ﬁve tasks: entity typing on the Open Entity dataset (Choi et al., 2018), relation classiﬁcation on the TACRED dataset (Zhang et al., 2017), NER on the CoNLL-2003 dataset (Tjong Kim Sang and De Meulder,…”
LEWIS: Levenshtein Editing for Unsupervised Text Style Transfer (2021, background): “This coarse-to-ﬁne transformations of text, in which the input context is progressively reﬁned, has also led to improvements in syntactic parsing (Charniak and Johnson, 2005), semantic parsing (Dong and Lapata, 2018), and NER (Choi et al., 2018).”
Retrieval And Structuring Augmented Generation with Large Language Models (2025, unknown): “Ultra-fine entity typing (UFET) [34] expanded the label space to thousands of types.”