Exposing the Implicit Energy Networks behind Masked Language Models via Metropolis-Hastings

By Kartik Goyal, Chris Dyer, Taylor Berg-Kirkpatrick. 2021

54 citations recorded by the source corpus.

While recent work has shown that scores from models trained by the ubiquitous masked language modeling (MLM) objective effectively discriminate probable from improbable sequences, it is still an open question if these MLMs specify a principled probability distribution over the space of possible sequences. In this paper, we interpret MLMs as energy-based sequence models and propose two energy parametrizations derivable from the trained MLMs. In order to draw samples correctly from these models, we develop a tractable sampling scheme based on the Metropolis--Hastings Monte Carlo algorithm. In our approach, samples are proposed from the same masked conditionals used for training the masked language models, and they are accepted or rejected based on their energy values according to the target distribution. We validate the effectiveness of the proposed parametrizations by exploring the quality of samples drawn from these energy-based models for both open-ended unconditional generation and a conditional generation task of machine translation. We theoretically and empirically justify our sampling algorithm by showing that the masked conditionals on their own do not yield a Markov chain whose stationary distribution is that of our target distribution, and our approach generates higher quality samples than other recently proposed undirected generation approaches (Wang et al., 2019, Ghazvininejad et al., 2019).

Subfields: Artificial Intelligence, Large Language Models, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Large Language Models

In Probabilistic & Bayesian Methods

Citations in the corpus

Aligning Language Models with Preferences through f-divergence Minimization (2023, methodology): “…such as rejection sampling to sample from simple distributions like pGDC bin (Roller et al., 2021; Ziegler et al., 2022), and Quasi Rejection Sampling (QRS) (Eikema et al., 2022) or MCMC techniques (Miao et al., 2019; Goyal et al., 2022) to sample from more complex distributions, such as pGDC dist.”
Mix and Match: Learning-free Controllable Text Generationusing Energy Language Models (2022, methodology): “…of these black-box experts as a probabilistic energy model (Hinton, 2002) – i.e., a non-autoregressive , globally normalized language model – and then sample (without further training or ﬁne-tuning) using a specialized Gibbs sampler with a Metropolis-Hastings correction step (Goyal et al., 2021).”
COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamics (2022, unknown): “The recent CGMH [35] and TSMH [57], followed by [36, 14], perform constrained decoding with extended Gibbs sampling or Metropolis-Hastings sampling in the discrete text space.”
An approximate sampler for energy-based models with divergence diagnostics (2022, methodology): “Goyal et al. (2021) develop a Metropolis-Hastings algorithm to sample from masked language models.”
Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models (2021, methodology): “values; these models can be viewed as Markov random fields and sampled using Gibbs/Metropolis Hastings via iterative sampling of the masked distributions [60], [238].”
Continuous diffusion for categorical data (2022, background): “1Although several works have explored generative approaches based on masked language models (Ghazvininejad et al., 2019; Goyal et al., 2021; Shih et al., 2022; Wang and Cho, 2019), they were originally introduced and are still mainly used for representation learning.”