Tokenizing, POS Tagging, Lemmatizing and Parsing UD 2.0 with UDPipe
By Milan Straka, Jana Straková. 2017
628 citations recorded by the source corpus.
Many natural language processing tasks, including the most advanced ones, routinely start by several basic processing steps – tokenization and segmentation, most likely also POS tagging and lemmatization, and commonly parsing as well. A multilingual pipeline performing these steps can be trained using the Universal Dependencies project, which contains annotations of the described tasks for 50 languages in the latest release UD 2.0. We present an update to UDPipe, a simple-to-use pipeline processing CoNLL-U version 2.0 files, which performs these tasks for multiple languages without requiring additional external data. We provide models for all 50 languages of UD 2.0, and furthermore, the pipeline can be trained easily using data in CoNLL-U format. UDPipe is a standalone application in C++, with bindings available for Python, Java, C# and Perl. In the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, UDPipe was the eight best system, while achieving low running times and moderately sized models.
Subfields: Artificial Intelligence, Natural Language Processing, Information Extraction, Computational Linguistics & Semantics
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Graham Neubig
- Noah A. Smith
- Yoav Goldberg
- Christopher D. Manning
- Yulia Tsvetkov
- Mohit Bansal
- Anders Søgaard
- Sebastian Ruder
In Natural Language Processing
- Yulia Tsvetkov
- Antonios Anastasopoulos
- Graham Neubig
- Yoav Goldberg
- Anders Søgaard
- Milan Straka
- Slav Petrov
- Martin Popel
In Information Extraction
- Slav Petrov
- Filip Ginter
- Anders Søgaard
- Adam Lopez
- Clara Vania
- Yova Kementchedjhieva
- Yulia Tsvetkov
- Tapio Salakoski
In Computational Linguistics & Semantics
- Slav Petrov
- Filip Ginter
- Anders Søgaard
- Adam Lopez
- Clara Vania
- Yova Kementchedjhieva
- Graham Neubig
- Yulia Tsvetkov
Citations in the corpus
- Evaluating the Morphosyntactic Well-formedness of Generated Texts (2021, methodology): “We experimented with commonly used multilingual parsers like UDPipe (Straka and Straková, 2017), UDify (Kondratyuk and Straka, 2019), and Stanza (Qi et al., 2020), settling on Stanza for its superior performance in preliminary experiments.”
- Edinburgh Research Explorer A systematic comparison of methods for low-resource dependency parsing on genuinely low-resource languages (positive): “2; Straka and Straková, 2017) and (2) the best system for each treebank, (3) our best approach, and (4) a cross-lingual model with fastText embeddings.”
- CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies (2018, methodology): “We prepared a set of baseline models using UD-Pipe 1.2 (Straka and Strakov´a, 2017).”
- Multilingual is not enough: BERT for Finnish (2019, methodology): “As the Udify model does not implement sentence or token segmentation, we use UDPipe (Straka and Straková, 2017) to pre-segment the text when reporting LAS on predicted segmentation.”