Neural Structural Correspondence Learning for Domain Adaptation

By Yftah Ziser, Roi Reichart. 2016

111 citations recorded by the source corpus.

We introduce a neural network model that marries together ideas from two prominent strands of research on domain adaptation through representation learning: structural correspondence learning (SCL, (Blitzer et al., 2006)) and autoencoder neural networks (NNs). Our model is a three-layer NN that learns to encode the non-pivot features of an input example into a low dimensional representation, so that the existence of pivot features (features that are prominent in both domains and convey useful information for the NLP task) in the example can be decoded from that representation. The low-dimensional representation is then employed in a learning algorithm for the task. Moreover, we show how to inject pre-trained word embeddings into our model in order to improve generalization across examples with similar pivot features. We experiment with the task of cross-domain sentiment classification on 16 domain pairs and show substantial improvements over strong baselines.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Roi Reichart
2. Alan Ramponi
3. Nadav Oved
4. Barbara Plank
5. Constantinos F. Karouzos
6. Georgios Paraskevopoulos
7. Alexandros Potamianos
8. Eyal Ben-David

Citations in the corpus

• The Hitchhiker’s Guide to Testing Statistical Significance in Natural Language Processing (2018, methodology): “This test is appropriate for binary classification tasks, and has been indeed used in such NLP works (e.g. sentiment classificaiton, (Blitzer et al., 2006; Ziser and Reichart, 2017)).”
• The Hitchhiker’s Guide to Computational Linguistics in Suicide Prevention (2021, unknown): “…scenarios, many efforts are directed today at domain adaptation research, that is, developing methods that can help adapt a model that was trained with data from one textual domain so that it can produce accurate predictions on data from other domains (e.g., Ziser & Reichart, 2017, 2018).”
• Neural Unsupervised Domain Adaptation in NLP—A Survey (2020, background): “A recent line of work (Ziser and Reichart, 2017; Ziser and Reichart, 2018a; Ziser and Reichart, 2018b; Ziser and Reichart, 2019) brings SCL back to neural networks.”
• Projecting Embeddings for Domain Adaption: Joint Modeling of Sentiment Analysis in Diverse Domains (2018, methodology): “Mutual Information Selected Pivots have been shown to be a good predictor of sentiment across domains (Blitzer et al., 2006; Pan et al., 2010; Ziser and Reichart, 2017).”
• Retrospective Reader for Machine Reading Comprehension (2020, unknown): “This test is designed for paired nominal observations, and it is appropriate for binary classification tasks (Ziser and Reichart 2017).”
• PADA: A Prompt-based Autoregressive Approach for Adaptation to Unseen Domains (2021, background): “A recent line of work married the two approaches and achieved impressive performance gains ( AE-SCL and PBLM , (Ziser and Reichart, 2017, 2018b)).”
• UDALM: Unsupervised Domain Adaptation through Language Modeling (2021, background): “Ziser and Reichart (2017, 2018, 2019) combine SCL with neural network architectures and language modeling.”
• Pivot Based Language Modeling for Improved Neural Domain Adaptation (2018, methodology): “The division of the feature set into pivots and non-pivots is based on the the method of ZR17 that followed the work of Blitzer et al. (2007) (details are in Appendix C).”