Language Identification: The Long and the Short of the Matter

By Timothy Baldwin, Marco Lui. 2010

186 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

Citations in the corpus

Learning Word Vectors for 157 Languages (2018, methodology): “We evaluate our model on publicly available datasets from Baldwin and Lui (2010) and report results in Table 2.”
A Fast, Compact, Accurate Model for Language Identification of Codemixed Text (2018, background): “Language ID of monolingual text has been extensively studied (Hughes et al., 2006; Baldwin and Lui, 2010; Lui and Baldwin, 2012; King and Abney, 2013), but language ID for codemixed text has received much less attention.”
langid.py: An Off-the-shelf Language Identification Tool (2012, methodology): “We compared the four systems on datasets used in previous language identification research (Baldwin and Lui, 2010) (EUROGOV, TCL, WIKIPEDIA ), as well as an extract from a biomedical parallel corpus (Tiedemann, 2009) (EMEA) and a corpus of samples from the Europarl Parallel Corpus (Koehn, 2005)…”
Computational Sociolinguistics: A Survey (2016, background): “Most studies taking this approach focus on automatic dialect identification, which is a variation of automatic language identification, a well-studied research topic within the field of computational linguistics (Baldwin and Lui 2010; Hughes et al. 2006).”
Code Mixing: A Challenge for Language Identification in the Language of Social Media (2014, background): “We follow in the footsteps of recent work on language identification for SMC (Hughes et al., 2006; Baldwin and Lui, 2010; Bergsma et al., 2012), focusing specifically on the problem of word-level language identification for code mixing SMC.”
Natural Language Processing with Small Feed-Forward Networks (2017, methodology): “We restrict our focus to single language identification across diverse languages, and compare to the work of Baldwin and Lui (2010) on predicting the language of Wikipedia text in 66 languages.”
Lexical Normalisation of Short Text Messages: Makn Sens a #twitter (2011, methodology): “7 The language ﬁltering of Twitter to automatically identify English tweets was based on the language identiﬁcation method of Baldwin and Lui (2010), using the EuroGOV dataset as training data, a mixed unigram/bigram/trigram byte feature representation, and a skew divergence nearest prototype…”
Spoken Language Recognition: From Fundamentals to Practice (2013, background): “KEYWORDS | Acoustic features; calibration; classifier; fusion; language recognition evaluation (LRE); phonotactic features; spoken language recognition; tokenization; vector space modeling”