OntoNotes: The 90% Solution

By Eduard H. Hovy, Mitchell P. Marcus, Martha Palmer, Lance A. Ramshaw, Ralph M. Weischedel. 2006

1,052 citations recorded by the source corpus.

We describe the OntoNotes methodology and its result, a large multilingual richly-annotated corpus constructed at 90% interannotator agreement. An initial portion (300K words of English newswire and 250K words of Chinese newswire) will be made available to the community during 2007.

Subfields: Artificial Intelligence, Molecular Biology, Natural Language Processing, Evaluation & Benchmarks

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Molecular Biology

In Natural Language Processing

In Evaluation & Benchmarks

Citations in the corpus

Grammar as a Foreign Language (2014, methodology): “We use the OntoNotes corpus version 5 (Hovy et al., 2006), the English Web Treebank (Petrov & McDonald, 2012) and the updated and correctedQuestion Treebank (Judge et al., 2006).1 Note that the popular Wall Street Journal section of the PennTreebank (Marcus et al., 1993) is part of the OntoNotes…”
Semi-Supervised Sequence Modeling with Cross-View Training (2018, methodology): “Fine-Grained NER (FGN): We use the OntoNotes (Hovy et al., 2006) dataset.”
SemEval-2017 Task 1: Semantic Textual Similarity Multilingual and Crosslingual Focused Evaluation (2017, background): “Significant research effort has focused on STS over English sentence pairs.2 English STS is a 1i.a., news headlines, video and image descriptions, glosses from lexical resources including WordNet (Miller, 1995; Fellbaum, 1998), FrameNet (Baker et al., 1998), OntoNotes (Hovy et al., 2006), web discus”
Underspecification Presents Challenges for Credibility in Modern Machine Learning (2020, methodology): “We obtain predictors for this task by fine-tuning BERT checkpoints on the OntoNotes dataset (Hovy et al., 2006).”
Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks (2015, methodology): “In this table we report the F1 score on the WSJ 22 development set [23].”
SemEval-2014 Task 10: Multilingual Semantic Textual Similarity (2014, methodology): “For OnWN , we used the sense deﬁnition pairs of OntoNotes (Hovy et al., 2006) and WordNet (Fellbaum, 1998).”
Datasets: A Community Library for Natural Language Processing (2021, background): “Projects like OntoNotes have collected annotations across multiple tasks for a single corpus (Pradhan and Xue, 2009) whereas the Universal Dependency treebank (Nivre et al., 2016) collects similar annotations across languages.”
Lessons from archives: strategies for collecting sociocultural data in machine learning (2019, background): “ears as the standard for Part-of-Speech tags [45]. Others include OntoNotes for coreference resolution and named entity recognition (NER) and the EuroParl parallel corpus for machine translation (MT) [30, 40]. Sometimes, researchers and practitioners build on top of existing datasets motivated by av”
LFPT5: A Unified Framework for Lifelong Few-shot Language Learning Based on Prompt Tuning of T5 (2021, methodology): “For NER, we use CoNLL03 (Sang & De Meulder, 2003) and OntoNotes (Hovy et al., 2006) as different domains.”
Active Bias: Training More Accurate Neural Networks by Emphasizing High Variance Samples (2017, methodology): “0 [20] datasets using the CNN from Strubell et al.”
Globally Normalized Transition-Based Neural Networks (2016, methodology): “, 1993) with standard POS tagging splits; (2) the English “Treebank Union” multi-domain corpus containing data from the OntoNotes corpus version 5 (Hovy et al., 2006), the English Web Treebank (Petrov and McDonald, 2012), and the updated and corrected Question Treebank (Judge et al., 2006) with iden”
PAQ: 65 Million Probably-Asked Questions and What You Can Do With Them (2021, methodology): “, 2020) NER system, trained on OntoNotes (Hovy et al., 2006).”
Question-Answer Driven Semantic Role Labeling: Using Natural Language to Annotate Natural Language (2015, background): “PropBank (Kings-bury and Palmer, 2002), NomBank (Meyers et al., 2004) and OntoNotes (Hovy et al., 2006) 3, 5, 6, 9), implicit or inferred relations (Ex 4, 7, 8, 9) and roles that are not deﬁned in PropBank (Ex 1, 5).”
Moments in Time Dataset: One Million Videos for Event Understanding (2018, background): “We then clustered the verbs according to their conceptual structure and meaning using the features for each verb from Propbank [28], FrameNet [5] and OntoNotes [21].”
FLAIR: An Easy-to-Use Framework for State-of-the-Art NLP (2019, background): “, 2018) de default PoS tagging Ontonotes (Hovy et al., 2006) en default, fast Semantic Frame Detection PropBank (Bonial et al.”
SemEval-2016 Task 1: Semantic Textual Similarity, Monolingual and Cross-Lingual Evaluation (2016, background): “, 1998), OntoNotes (Hovy et al., 2006), web discussion forums, and Q&A data sets).”
Latent Multi-Task Architecture Learning (2017, background): “2013), not only due to its high inter-annotator agreement (Hovy et al. 2006), but also because it enables us to analyze the generalization ability of our models across different tasks and domains.”