Contextual String Embeddings for Sequence Labeling

By Alan Akbik, Duncan A. J. Blythe, Roland Vollgraf. 2018

1,382 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Natural Language Processing

Citations in the corpus

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (2019, neutral): “6 CSE (Akbik et al., 2018) - 93.”
HuggingFace's Transformers: State-of-the-art Natural Language Processing (2019, neutral): “Transformers is also included either as a dependency or with a wrapper in several popular NLP frameworks such as Spacy (Honnibal and Montani, 2017), AllenNLP (Gardner et al., 2018) or Flair (Akbik et al., 2018).”
HuggingFace's Transformers: State-of-the-art Natural Language Processing (2019, neutral): “ibrary1. Transformers is also included either as a dependency or with a wrapper in several popular NLP frameworks such as Spacy (Honnibal and Montani, 2017), AllenNLP (Gardner et al., 2018) or Flair (Akbik et al., 2018). Transformers is an ongoing effort maintained by the team of engineers and resea”
The People's Speech: A Large-Scale Diverse English Speech Recognition Dataset for Commercial Usage (2021, methodology): “We also ran the “Ontonotes fast model” for NER [13] on the speech transcripts and textual metadata to measure the diversity of the dataset in terms of the most discussed topics, etc.”
Unsupervised Cross-lingual Representation Learning at Scale (2019, methodology): “Without the CRF, our XLM-R model still performs on par with the state of the art, outperforming Akbik et al. (2018) on Dutch by 2 .”
Man is to Person as Woman is to Location: Measuring Gender Bias in Named Entity Recognition (2019, methodology): “5 0.06 0.07 0.08 0.09 0.1 1875 1895 1915 1935 1955 1975 1995 2015 Flair female male 0.58 0.6 0.62 0.64 0.66 0.68 0.7 0.72 1875 1895 1915 1935 1955 1975 1995 2015 Small Spacy female male 0 0.01 0.02 0.03 0.04 0.05 0.06 1875 1895 1915 1935 1955 1975 1995 2015 Flair female male 0 0.05 0.1 0.15 0.2 0.25”
FLAIR: An Easy-to-Use Framework for State-of-the-Art NLP (2019, background): “Similarly, recent works—including our own—have proposed meth-ods that produce different embeddings for the same word depending on its contextual usage (Peters et al., 2018a; Akbik et al., 2018; Devlin et al., 2018).”
Pre-trained models for natural language processing: A survey (2020, methodology): “abel corresponds to one word. The model is used to predict the label of each word. Since ELMo and BERT have shown their power in NLP, there is much work about pre-trained models for NER. Akbik et al. [1] used a pre-trained character-level language model to produce word-level embedding for NER. TagLM”
Stanza: A Python Natural Language Processing Toolkit for Many Human Languages (2020, methodology): “For NER we adopt the contextualized string representation-based sequence tagger from Akbik et al. (2018).”
A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages (2020, methodology): “…and language models have been developed using both feature-based architectures, the most notable examples being ELMo and Flair (Peters et al., 2018; Akbik et al., 2018), and trans-former based architectures, that are commonly used in a ﬁne-tune setting, as is the case of GPT-1, GPT-2 (Radford et…”
SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020) (2020, methodology): “…n -grams character n -grams lexicons for sentiment analysis and lexicon of offensive words Other representations included emoji priors extracted from the weakly supervised dataset, and sentiment analysis using NLTK (Bird et al., 2009), Vader (Hutto and Gilbert, 2014) and FLAIR (Akbik et al., 2018).”
CamemBERT: a Tasty French Language Model (2019, background): “Here the recent efforts of (Strakov´a et al., 2019) settled the state of the art for Spanish and Dutch, while (Akbik et al., 2018) did it for German.”
REL: An Entity Linker Standing on the Shoulders of Giants (2020, methodology): “REL has been developed with the following design considerations: • Use state-of-the-art approaches for entity disambiguation (ED) [8, 15] and named entity recognition (NER) [1], ensuring it is on par with the state-of-the-art on end-to-end entity linking [14].”
Is ChatGPT a General-Purpose Natural Language Processing Task Solver? (2023, background): “5 ( zero-shot ) and recent state-of-the-art named entity recognition models including Flair (Akbik et al., 2018), LUKE ( fine-tuned ) (Yamada et al., 2020), and ACE ( fine-tuned ) (Wang et al., 2020).”
German’s Next Language Model (2020, methodology): “For GermEval14 we report the result that can be achieved using the FLAIR framework (Akbik et al., 2019).”
Improving Named Entity Recognition by External Context Retrieving and Cooperative Learning (2021, methodology): “Most of the work takes NER as a sequence labeling problem and applies the linear-chain CRF (Lafferty et al., 2001) to achieve state-of-the-art accuracy (Ma and Hovy, 2016; Lample et al., 2016; Akbik et al., 2018, 2019; Wang et al., 2020b).”
Portuguese Named Entity Recognition using BERT-CRF (2019, methodology): “n performed by the CRF layer. Recent works explored contextual embeddings extracted from language models in conjunction with the LSTM-CRF architecture.Santos et al. (2019b,a) employ Flair Embeddings (Akbik et al., 2018) to extract contextual word embeddings from a bidirectional character-level LM tr”
Transformer models for text-based emotion detection: a review of BERT-based approaches (2021, methodology): “However, the model’s performance could have been improved by introducing additional contextual embeddings such as ELMo and FLAIR (Akbik et al. 2018) to test the performance of the BERT-Emotion detection.”