A Bayesian Framework for Information-Theoretic Probing
By Tiago Pimentel, Ryan Cotterell. 2021
27 citations recorded by the source corpus.
Pimentel et al. (2020) recently analysed probing from an information-theoretic perspective. They argue that probing should be seen as approximating a mutual information. This led to the rather unintuitive conclusion that representations encode exactly the same information about a target task as the original sentences. The mutual information, however, assumes the true probability distribution of a pair of random variables is known, leading to unintuitive results in settings where it is not. This paper proposes a new framework to measure what we term Bayesian mutual information, which analyses information from the perspective of Bayesian agents—allowing for more intuitive findings in scenarios with finite data. For instance, under Bayesian MI we have that data can add information, processing can help, and information can hurt, which makes it more intuitive for machine learning applications. Finally, we apply our framework to probing where we believe Bayesian mutual information naturally operationalises ease of extraction by explicitly limiting the available background knowledge to solve a task.
Subfields: Artificial Intelligence, Computer Science Applications
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
- Natasha Seelam
- Thomas McGrath
- Ulrich Paquet
- Oleg Serikov
- Miruna Clinciu
- Adam Pearce
- Ryan Teehan
- Vladimir Kramnik
In Computer Science Applications
Citations in the corpus
- Understanding Dataset Difficulty with V-Usable Information (2021, neutral): “V -information (Xu et al., 2019) as well as to condition out information for probing-based interpretability techniques (Hewitt et al., 2021; Pimentel & Cotterell, 2021).”
- Acquisition of chess knowledge in AlphaZero (2021, methodology): “More sophisticated regression methods such as data-dependent sparsity using Gated Linear Networks [84], information-theoretic regularisation using the information bottleneck [85], minimum description length probing [86] or Bayesian probing [87] may provide ways to mitigate this problem.”
- Emergent Structures and Training Dynamics in Large Language Models (2022, background): “…Probing Information-202 theoretic probing characterize tasks as a way of esti-203 mating the mutual information between an internal representation and the linguistic property of inter-205 est (Pimentel et al., 2020b; Pimentel and Cotterell, 2021; Voita and Titov, 2020; Pimentel et al., 2020a).”