Farasa: A Fast and Furious Segmenter for Arabic

By Ahmed Abdelali, Kareem Darwish, Nadir Durrani, Hamdy Mubarak. North American Chapter of the Association for Computational Linguistics, 2016

400 citations recorded by the source corpus.

In this paper, we present Farasa, a fast and accurate Arabic segmenter. Our approach is based on SVM-rank using linear kernels. We measure the performance of the seg-menter in terms of accuracy and efﬁciency, in two NLP tasks, namely Machine Translation (MT) and Information Retrieval (IR). Farasa outperforms or is at par with the state-of-the-art Arabic segmenters (Stanford and MADAMIRA), while being more than one order of magnitude faster.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Language and Linguistics

View Paper: https://www.semanticscholar.org/paper/a1aaa7c75464e7ebe41cfe5c5258241ca34c6414

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

In Language and Linguistics

Citations in the corpus

ARBERT & MARBERT: Deep Bidirectional Transformers for Arabic (2020, methodology): “The second version of AraBERT, AraBERTv1, employs the Farasa tool (Abdelali et al., 2016) to segment input text before training.”
The Interplay of Variant, Size, and Task Type in Arabic Pre-trained Language Models (2021, methodology): “We do not use models that require morphological pre-tokenization to allow direct comparison, and also because existing tokenization systems are mostly focused on MSA or EGY (Pasha et al., 2014; Abdelali et al., 2016; Obeid et al., 2020).”
AraBERT: Transformer-based Model for Arabic Language Understanding (2020, methodology): “To avoid this issue, we first segment the words using Farasa (Abdelali et al., 2016) into stems, prefixes and suf-”
On the Linguistic Representational Power of Neural Machine Translation Models (2019, background): “is often addressed by splitting tokens in morphologically-rich languages into constituents in a preprocessing step, using word segmentation in Arabic (Pasha et al. 2014; Abdelali et al. 2016) or compound splitting in German (Koehn and Knight 2003).”
Challenging Language-Dependent Segmentation for Arabic: An Application to Machine Translation and Part-of-Speech Tagging (2017, methodology): “Here we use the state-of-the-art Arabic segmenter MADAMIRA and Farasa as our baselines.”
How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models (2020, methodology): “The only difference between them is that the pretraining data for version 1 was pre-segmented with Farasa (Abdelali et al., 2016) before applying WordPiece.”