Learning Dynamic Author Representations with Temporal Language Models

By Edouard Delasalles, Sylvain Lamprier, Ludovic Denoyer. 2019

10 citations recorded by the source corpus.

Language models are at the heart of numerous works, notably in the text mining and information retrieval communities. These statistical models aim at extracting word distributions, from simple unigram models to recurrent approaches with latent variables that capture subtle dependencies in texts. However, those models are learned from word sequences only, and authors' identities, as well as publication dates, are seldom considered. We propose a neural model, based on recurrent language modeling, which aims at capturing language diffusion tendencies in author communities through time. By conditioning language models with author and temporal vector states, we are able to leverage the latent dependencies between the text contexts. This allows us to beat several temporal and non-temporal language baselines on two real-world corpora, and to learn meaningful author representations that vary through time.

Subfields: Artificial Intelligence, Natural Language Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Natural Language Processing

Citations in the corpus

Time-Aware Language Models as Temporal Knowledge Bases (2021, methodology): “We parametrize the masked language modeling objective (MLM; Devlin et al., 2019) with temporal information, P ( y | x, t ; θ ) , where y is a masked token or span, x is the textual context, and t is the time (§ 2.3).”
Alternate Endings: Improving Prosody for Incremental Neural TTS with Predicted Future Text Input (2021, background): “Further prediction gains could be achieved if the language model was fine-tuned on the traits of a specific author [31]; this would be an advisable step in the use case of assistive technologies for the speech impaired.”