Fisher Information for Distributed Estimation under a Blackboard Communication Protocol

By Leighton Pate Barnes, Yanjun Han, Ayfer Özgür. 2019

29 citations recorded by the source corpus.

We consider the problem of learning high-dimensional discrete distributions and structured (e.g. Gaussian) distributions in distributed networks, where each node in the network observes an independent sample from the underlying distribution and can use k bits to communicate its sample to a central processor. We consider a blackboard communication model, where nodes can share information interactively through a public blackboard but each node is restricted to write at most k bits on the final transcript. We characterize the impact of the communication constraint k on the minimax risk of estimating the underlying distribution under ℓ2 loss, and develop minimax lower bounds that apply in a unified way to many common statistical models. This is achieved by explicitly characterizing the Fisher information from the blackboard transcript.

Subfields: Artificial Intelligence, Computer Networks and Communications, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Networks and Communications

In Statistics and Probability

Citations in the corpus

rTop-k: A Statistical Estimation Approach to Distributed SGD (2020, methodology): “To prove Theorem 2, we use part of Theorem 3 from [50] (see also [36]), reproduced below, which is proved using a Fisher information argument.”
Learning Distributions from their Samples under Communication Constraints (2019, background): “Equivalently, we ask the question: how can we best represent X with k bits so as to maximize the Fisher information it provides about θ? This framework was first introduced by the authors in [1],[2].”
Unified lower bounds for interactive high-dimensional estimation under information constraints (2020, background): “The average information bound in (3), in fact, allows us to recover bounds similar to the van Trees inequality-based bounds developed in [8] and [7].”