Stanza: A Python Natural Language Processing Toolkit for Many Human Languages

By Peng Qi, Yuhao Zhang, Yuhui Zhang, Jason Bolton, Christopher D. Manning. 2020

2,124 citations recorded by the source corpus.

We introduce Stanza, an open-source Python natural language processing toolkit supporting 66 human languages. Compared to existing widely used toolkits, Stanza features a language-agnostic fully neural pipeline for text analysis, including tokenization, multi-word token expansion, lemmatization, part-of-speech and morphological feature tagging, dependency parsing, and named entity recognition. We have trained Stanza on a total of 112 datasets, including the Universal Dependencies treebanks and other multilingual corpora, and show that the same neural architecture generalizes well and achieves competitive performance on all languages tested. Additionally, Stanza includes a native Python interface to the widely used Java Stanford CoreNLP software, which further extends its functionality to cover other tasks such as coreference resolution and relation extraction. Source code, documentation, and pretrained models for 66 languages are available at https://stanfordnlp.github.io/stanza/.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

End-to-End Training of Multi-Document Reader and Retriever for Open-Domain Question Answering (2021, methodology): “We make use of the Stanza toolkit (Qi et al., 2020) to segment evidence documents into sentences.”
The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset (2023, methodology): “split_sentences_{lang} Builds a sentence splitter depending on the language passed: For Arabic, Catalan, Basque, Indonesian, and Chinese (both simplified and traditional), we use the Stanza tokenizer (Qi et al., 2020).”
End-to-End Training of Neural Retrievers for Open-Domain Question Answering (2021, methodology): “We use the pre-trained models provided by the Stanza toolkit (Qi et al., 2020) to segment Wikipedia paragraphs into sentences and extract named entities.”
Investigating Failures of Automatic Translation in the Case of Unambiguous Gender (2021, background): “…measuring performance for unambiguous examples, it does not focused on the NMT use case, and its examples require cross-sentential coreferences, which will likely require a more complex linguistic toolbox than our intrasentential case (Szabolcsi, 2003; Hardmeier and Federico, 2010; Reinhart, 2016).”
CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities (2022, methodology): “• Engine: davinci • Response length (word piece): 30 • Temperature: 0.2, 0.3, 0.75, 0.9 • Top P: 1 • Frequency penalty: 0, 0.5, 1 • Presence penalty: 0 • Best of: 1 A.3 Suggestions Once GPT-3 generated an output given a prompt, it was parsed as a list of sentences using Stanza [44], and only the fir”
Improving Cross-Modal Alignment in Vision Language Navigation via Syntactic Information (2021, methodology): “We use Stanza Toolkit (Qi et al., 2020a) to generate the dependency parse tree for Hindi and Telugu.”
Perturbation Augmentation for Fairer NLP (2022, methodology): “Since word lists are limited in coverage, we also use the Stanza Named Entity Recognition (NER) module (Qi et al., 2020) to identify named entities in text.”
Visually Grounded Continual Learning of Compositional Phrases (2020, methodology): “First, we append part-of-speech tags (POS) to each caption using Stanza (Qi et al., 2020).”
Dependency Parsing based Semantic Representation Learning with Graph Neural Network for Enhancing Expressiveness of Text-to-Speech (2021, methodology): “Following GraphSpeech, we use Stanza [28] to extract dependency trees.”
Evaluating the Morphosyntactic Well-formedness of Generated Texts (2021, methodology): “2017), UDify (Kondratyuk and Straka, 2019), and Stanza (Qi et al., 2020), settling on Stanza for its superior performance in preliminary experiments.”
Joint Learning of Localized Representations from Medical Images and Reports (2021, methodology): “We split the extracted text into sentences using Stanza[62] again.”
Large language models are few-shot clinical information extractors (2022, methodology): “We tokenized using Stanza (Qi et al., 2020) for these metrics.”
Graphspeech: Syntax-Aware Graph Attention Network for Neural Speech Synthesis (2020, methodology): “In this study, we use Stanza [30] to extract the syntactic dependency parser trees.”