Supervised Learning Under Distributed Features

By Bicheng Ying, Kun Yuan, Ali H. Sayed. 2018

44 citations recorded by the source corpus.

This paper studies the problem of learning under both large datasets and large-dimensional feature space scenarios. The feature information is assumed to be spread across agents in a network, where each agent observes some of the features. Through local cooperation, the agents are supposed to interact with each other to solve an inference problem and converge towards the global minimizer of an empirical risk. We study this problem exclusively in the primal domain, and propose new and effective distributed solutions with guaranteed convergence to the minimizer with linear rate under strong convexity. This is achieved by combining a dynamic diffusion construction, a pipeline strategy, and variance-reduced techniques. Simulation results illustrate the conclusions.

Subfields: Computer Networks and Communications, Modeling and Simulation

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Networks and Communications

In Modeling and Simulation

Citations in the corpus

Learning Privately over Distributed Features: An ADMM Sharing Approach (2019, methodology): “A few recent studies [8, 9, 10, 11, 12, 13, 14] have revisited vertically partitioned features under the setting of distributed learning, which is motivated by the increase in feature dimensionality as well as the opportunity of cooperation between multiple parties that may hold different aspects of”
A Sharp Estimate on the Transient Time of Distributed Stochastic Gradient Descent (2019, background): “gent i only, and all the agents communicate and exchange information over a network. Problems in the form of (1.1) ﬁnd applications in multi-agent target seeking [31, 8], distributed machine learning [13, 24, 10, 2, 45, 1, 4], and wireless networks [9, 20, 2], among other scenarios. In order to solv”
FedBCD: A Communication-Efficient Collaborative Learning Framework for Distributed Features (2022, background): “[19] propose a dynamic diffusion approach which requires communications between parties with its related neighbours.”
A Non-Asymptotic Analysis of Network Independence for Distributed Stochastic Gradient Descent (2019, background): “1) find applications in multi-agent target seeking [32, 8], distributed machine learning [13, 24, 10, 2, 46, 1, 4], and wireless networks [9, 20, 2], among other scenarios.”
Edge Artificial Intelligence for 6G: Vision, Enabling Technologies, and Applications (2021, methodology): “Consensus algorithms were also developed in [158] to jointly learn a model under the decentralized network while keeping the distributed data features locally.”