A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios

By Michael A. Hedderich, Lukas Lange, Heike Adel, Jannik Strotgen, Dietrich Klakow. North American Chapter of the Association for Computational Linguistics, 2020

396 citations recorded by the source corpus.

Deep neural networks and huge language models are becoming omnipresent in natural language applications. As they are known for requiring large amounts of training data, there is a growing body of work to improve the performance in low-resource settings. Motivated by the recent fundamental changes towards neural models and the popular pre-train and fine-tune paradigm, we survey promising approaches for low-resource natural language processing. After a discussion about the different dimensions of data availability, we give a structured overview of methods that enable learning when training data is sparse. This includes mechanisms to create additional labeled data like data augmentation and distant supervision as well as transfer learning settings that reduce the need for target supervision. A goal of our survey is to explain how these methods differ in their requirements as understanding them is essential for choosing a technique suited for a specific low-resource setting. Further key aspects of this work are to highlight open issues and to outline promising directions for future research.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

View Paper: https://www.semanticscholar.org/paper/455cdafd55a5b5ddefa029bf97801327e142646d

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

An Empirical Survey of Data Augmentation for Limited Data Learning in NLP (2021, background): “Paraphrasing has been widely adopted as a data augmentation technique in various NLP tasks (Yu et al., 2018; Xie et al., 2020; Kumar et al., 2019; He et al., 2020; Chen et al., 2020b,c; Cai et al., 2020), as it generally provides more diverse augmented text with different word choices and sentence s”
mGPT: Few-Shot Learners Go Multilingual (2022, background): “However, there are several less explored directions in the interconnected areas of zero/few-shot learning and multilingual NLP, including model development for typologically distant and low-resource languages underrepresented in LMs’ pre-training corpora (Wu and Dredze, 2020; Lauscher et al., 2020; ”
SwitchPrompt: Learning Domain-Specific Gated Soft Prompts for Classification in Low-Resource Domains (2023, background): “We refer to the survey of Hedderich et al. (2021) for more information on language model adaptation for low-resource domains.”
A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasets (2023, neutral): “, 2022d), Low-Resourced (Hedderich et al., 2021) NLP problems, Data-to-Text Generation (Kantharaj et al.”
Neural Machine Translation for Low-resource Languages: A Survey (2021, methodology): “In Natural Language Processing (NLP), a low-resource problem can arise mainly due to the considered languages being low-resourced or the considered domains being low-resourced [78].”