Introduction to the CoNLL-2005 Shared Task: Semantic Role Labeling
By Xavier Carreras, Lluís Màrquez i Villodre. 2005
778 citations recorded by the source corpus.
In this paper we describe the CoNLL-2005 shared task on Semantic Role Labeling. We introduce the specification and goals of the task, describe the data sets and evaluation methods, and present a general overview of the 19 systems that have contributed to the task, providing a comparative description and results.
Subfields: Artificial Intelligence, Molecular Biology, Natural Language Processing, Evaluation & Benchmarks, Information Extraction
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Luke S. Zettlemoyer
- Mohit Bansal
- Noah A. Smith
- Percy Liang
- Christopher D. Manning
- Jianfeng Gao
- Graham Neubig
- Yejin Choi
In Molecular Biology
- Luke S. Zettlemoyer
- Andrew McCallum
- Bishan Yang
- Kenton C. T. Lee
- Luheng He
- Noah A. Smith
- Tom Michael Mitchell
- Junichi Tsujii
In Natural Language Processing
- Luke S. Zettlemoyer
- Mohit Bansal
- Noah A. Smith
- Christopher D. Manning
- Percy Liang
- Dan Roth
- Yoav Goldberg
- Graham Neubig
In Evaluation & Benchmarks
- Luke S. Zettlemoyer
- Mohit Bansal
- Noah A. Smith
- Omer Levy
- Dan Roth
- Wen-tau Yih
- Alessandro Moschitti
- Andrew McCallum
In Information Extraction
- Dan Roth
- Aliaksei Severyn
- Lluís Màrquez i Villodre
- Alessandro Moschitti
- Tom Michael Mitchell
- Dipanjan Das
- Bishan Yang
- Luke S. Zettlemoyer
Citations in the corpus
- GLM-130B: An Open Bilingual Pre-trained Model (2022, methodology): “We create two different tasks based on semantic role labelling datasets CoNLL05 (Carreras & Màrquez, 2005), CoNLL12 (Pradhan et al., 2013), and PropBank (Kingsbury & Palmer).”
- LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention (2023, methodology): “Instruction-following Evaluation Following Stanford Alpaca [60], we utilize We fine-tune LLaMA-Adapter on 8 A100 GPUs for 5 epochs.”
- End-to-end learning of semantic role labeling using recurrent neural networks (2015, methodology): “The test set consists of section 23 of WSJ concatenated with 3 sections from Brown corpus (Carreras and Màrquez, 2005).”
- Deep Semantic Role Labeling: What Works and What’s Next (2017, methodology): “While the CoNLL 2005 shared task assumes gold predicates as input (Carreras and Màrquez, 2005), this information is not available in many downstream applications.”
- P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks (2021, methodology): “For multitask setting, the pre-train training set is a combination of the training set of CoNLL05 (Carreras and Màrquez, 2005), CoNLL12 (Pradhan et al.”
- Linguistically-Informed Self-Attention for Semantic Role Labeling (2018, positive): “We present results on the CoNLL-2005 shared task (Carreras and Marquèz, 2005) and the CoNLL-2012 English subset of OntoNotes 5.0 (Pradhan et al., 2006), achieving state-of-the-art results for a single model with predicted predicates on both corpora. In all experiments, we initialize with pre-trained”
- Visual Semantic Role Labeling (2015, background): “Semantic Role Labeling in a Natural Language Processing context refers to labeling words in a sentence with different semantic roles for the verb in the sentence [5].”
- Dependency-based Semantic Role Labeling of PropBank (2008, methodology): “…NLP community has recently devoted much attention to developing accurate and robust methods for performing role-semantic analysis automatically, and a number of multi-system evaluations have been carried out (Litkowski, 2004; Carreras and Màrquez, 2005; Baker et al., 2007; Surdeanu et al., 2008).”
- Efficient Inference in Large Conditional Random Fields (2006, methodology): “We adopt the CoNLL task specification [11], where a predicted syntactic parse tree is provided.”
- Deep Semantic Role Labeling with Self-Attention (2017, methodology): “The test set consists of section 23 of the WSJ corpus as well as 3 sections from the Brown corpus (Carreras and M`arquez 2005).”
- Greedy, Joint Syntactic-Semantic Parsing with Stack LSTMs (2016, background): “Though they did not achieve the best results on the constituent-based SRL task (Carreras and Màrquez, 2005), their approach inspired Zhou and Xu (2015), who achieved state-of-the-art results using deep bidirectional LSTMs.”
- Jointly Predicting Predicates and Arguments in Neural Semantic Role Labeling (2018, methodology): “We experiment on the CoNLL 2005 (Carreras and Màrquez, 2005) and CoNLL 2012 (OntoNotes 5.”
- Frame-Semantic Parsing (2014, methodology): “First, we will briefly discuss work done on PropBank-style semantic role labeling, following which we will concentrate on the more relevant problem of frame-semantic structure extraction.”