Protocols for Learning Classifiers on Distributed Data

By Hal Daumé Iii, Jeff M. Phillips, Avishek Saha, Suresh Venkatasubramanian. 2012

29 citations recorded by the source corpus.

We consider the problem of learning classifiers for labeled data that has been distributed across several nodes. Our goal is to find a single classifier, with small approximation error, across all datasets while minimizing the communication between nodes. This setting models real-world communication bottlenecks in the processing of massive distributed datasets. We present several very general sampling-based solutions as well as some two-way protocols which have a provable exponential speed-up over any one-way protocol. We focus on core problems for noiseless data distributed across two or more nodes. The techniques we introduce are reminiscent of active learning, but rather than actively probing labels, nodes actively communicate with each other, each node simultaneously learning the important data from another node.

Subfields: Artificial Intelligence, Computational Theory and Mathematics

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computational Theory and Mathematics

Citations in the corpus

Geometric Lower Bounds for Distributed Parameter Estimation Under Communication Constraints (2018, background): “…also an increasing number of applications in which data is generated in a distributed manner and it (or features of it) are communicated over bandwidth-limited links to central processors Boyd et al. (2011); Balcan et al. (2012); Daume III et al. (2012); Daumé et al. (2012); Dekel et al. (2012).”
A Distributed Frank-Wolfe Algorithm for Communication-Efficient Sparse Learning (2014, background): “From a fundamental perspective, studying the tradeoff between communication complexity [3, 4] and learning/optimization error has recently attracted some interest [5, 6, 7].”
Communication-Efficient Distributed Learning of Discrete Probability Distributions (2017, background): “In recent years, a number of statistical estimation problems have been algorithmically studied in the distributed setting [3, 16, 15, 40, 21, 30, 24, 33, 5, 29].”
Trading Computation for Communication: Distributed Stochastic Dual Coordinate Ascent (2013, background): “2 Related Work Recent years have seen the great emergence of distributed algorithms for solving machine learning related problems [3, 10].”