Semi-Supervised Learning and Domain Adaptation in Natural Language Processing
By Anders Søgaard. 2013
42 citations recorded by the source corpus.
This book introduces basic supervised learning algorithms applicable to natural language processing (NLP) and shows how the performance of these algorithms can often be improved by exploiting the marginal distribution of large amounts of unlabeled data. One reason for that is data sparsity, i.e., the limited amounts of data we have available in NLP. However, in most real-world NLP applications our labeled data is also heavily biased. This book introduces extensions of supervised learning algorithms to cope with data sparsity and different kinds of sampling bias. This book is intended to be both readable by first-year students and interesting to the expert audience. My intention was to introduce what is necessary to appreciate the major challenges we face in contemporary NLP related to data sparsity and sampling bias, without wasting too much time on details about supervised learning algorithms or particular NLP applications. I use text classification, part-of-speech tagging, and dependency parsing as running examples, and limit myself to a small set of cardinal learning algorithms. I have worried less about theoretical guarantees ("this algorithm never does too badly") than about useful rules of thumb ("in this case this algorithm may perform really well"). In NLP, data is so noisy, biased, and non-stationary that few theoretical guarantees can be established and we are typically left with our gut feelings and a catalogue of crazy ideas. I hope this book will provide its readers with both. Throughout the book we include snippets of Python code and empirical evaluations, when relevant. Table of Contents: Introduction / Supervised and Unsupervised Prediction / Semi-Supervised Learning / Learning under Bias / Learning under Unknown Bias / Evaluating under Bias
Subfields: Artificial Intelligence, Natural Language Processing
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Roi Reichart
- Jacob Eisenstein
- Diyi Yang
- Matthew E. Peters
- Noah A. Smith
- Amir Feder
- Mohit Bansal
- Kyunghyun Cho
In Natural Language Processing
- Matthew E. Peters
- Katherine A. Keith
- Jacob Eisenstein
- Diyi Yang
- Amir Feder
- Roi Reichart
- Brandon M Stewart
- Margaret E. Roberts
Citations in the corpus
- Semi-supervised sequence tagging with bidirectional language models (2017, background): “A detailed discussion of semi-supervised learning methods in NLP can be found in (Søgaard, 2013).”
- Mind the Class Weight Bias: Weighted Maximum Mean Discrepancy for Unsupervised Domain Adaptation (2017, background): “Second, the relationship between semi-supervised learning and domain adaptation has been studied in [33].”
- Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond (2021, background): “Features that are spuriously correlated with Y will be 2 From the perspective of earlier work on domain adaptation (Søgaard, 2013), spurious correlations can be viewed as a special case of a more general phenomenon in which feature-label relationships change across domains.”