Neural Temporality Adaptation for Document Classification: Diachronic Word Embeddings and Domain Adaptation Models

By Xiaolei Huang, Michael J. Paul. 2019

41 citations recorded by the source corpus.

Language usage can change across periods of time, but document classifiers models are usually trained and tested on corpora spanning multiple years without considering temporal variations. This paper describes two complementary ways to adapt classifiers to shifts across time. First, we show that diachronic word embeddings, which were originally developed to study language change, can also improve document classification, and we show a simple method for constructing this type of embedding. Second, we propose a time-driven neural classification model inspired by methods for domain adaptation. Experiments on six corpora show how these methods can make classifiers more robust over time.

Subfields: Artificial Intelligence, Natural Language Processing, Representation & Self-Supervised Learning

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

In Representation & Self-Supervised Learning

Citations in the corpus

Time-Aware Language Models as Temporal Knowledge Bases (2021, background): “Several studies have focused on degradation of models on test data from a different time period than their training data (Huang and Paul, 2018, 2019; Jaidka et al., 2018; Lukes and Søgaard, 2018; Florio et al., 2020).”
Temporal Effects on Pre-trained Models for Language Processing Tasks (2021, background): “of old and new data, time-aware self-attention and continual learning as new data is available (He et al., 2018; Huang and Paul, 2019; Bjerva et al., 2020; Hofmann et al., 2021; Rosin and Radinsky, 2022).”
RealTime QA: What's the Answer Right Now? (2022, background): “…and evaluation/deployment on many NLP tasks (Lazaridou et al., 2021; Röttger and Pierrehumbert, 2021; Luu et al., 2022; Onoe et al., 2022) and proposed mitigation methods for temporal adaptation (Huang and Paul, 2018, 2019; Dhingra et al., 2022; Jang et al., 2022a,b; Lee et al., 2022).”
Improving Document-Level Sentiment Classification Using Importance of Sentences (2020, methodology): “For fair comparison with the previous models, we encoded review scores of the Yelp-hotel dataset, the Yelp-rest dataset, and the Amazon dataset into three discrete categories (score >3 as positive, =3 as neutral, and <3 as negative) according to Huang and Paul’s experimental settings [29].”