Adaptation, Learning, and Optimization over Networks

By Ali H. Sayed. 2014

827 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

Robust Aggregation for Federated Learning (2019, background): “From the lens of learning in networks [77], federated learning comprises a star network where agents (i.”
Walkman: A Communication-Efficient Random-Walk Algorithm for Decentralized Optimization (2018, methodology): “Based on Theorem 1 and following the derivation of Section B2 of [24], with algorithmic parameter μi,j = 1 and all the edges uniformly selected with probability 1/m[2], the communication complexity of ESDACD to achieve the deviation Gt ≤ is”
Distributed multi-task classification: a decentralized online learning approach (2017, methodology): “It’s apparently more favorable to distribute smaller subsets of data to different learning units and then collaborate with direct neighbors to find the optimal solution [31].”
Communication-Censored ADMM for Decentralized Consensus Optimization (2019, methodology): “algorithms where a node can utilize its local gradient during the optimization process, such as gradient descent method [20], diffusion method [21], exact first-order algorithm [22], and linearized ADMM [23], [24]; (iii) second-order algorithms where a node can compute or approximately compute its l”
Initialization-free distributed algorithms for optimal resource allocation with feasibility constraints and application to economic dispatch of power systems (2015, methodology): “To circumvent the requirement of control center or master, various distributed optimization models or algorithms have been developed (Nedic, Ozdaglar, & Parrilo (2010), Sayed (2014), Lou, Shi, Johansson, & Hong (2014), and Yi & Hong (2014)).”
Multiagent Fully Decentralized Value Function Learning With Linear Convergence Rates (2018, methodology): “In view of the Perron–Frobenius theorem [32], this condition guarantees that the Markov Chain under φ(a|s) will have a steady-state distribution in which every state has a strictly positive probability of visitation [32].”
Adaptive Networks (2014, background): “These discoveries are motivating greater efforts toward a deeper examination of information processing over graphs in several disciplines including signal processing, machine learning, optimization, and control (see, e.g., [1], [9], and the references therein).”
A Proximal Gradient Algorithm for Decentralized Composite Optimization (2015, background): “Pioneered by the seminal work [9], [10] in 1980s, decentralized optimization, control, and decision-making in networked multi-agent systems have attracted increasing interest in recent years due to the rapid development of communication and computation technologies [11]–[13].”
Distributed Gradient Methods for Convex Machine Learning Problems in Networks: Distributed Optimization (2020, background): “One can find an elegant exposure of these strategies in [9] and follow the references therein for technical details of their analysis.”
Excess-Risk of Distributed Stochastic Learners (2013, background): “Now, in place of the non-cooperative strategy, let us assume that the N nodes transmit their samples to a central processor, which executes the following algorithm: wi = wi−1− µ(i) N N∑ k=1 ∇wQ(wi−1,xk,i) [unweighted cent.]”
NEXT: In-Network Nonconvex Optimization (2016, methodology): “…belonging to the former class include: i) consensus-based (sub)gradient schemes [1]–[4] along with their accelerated and asynchronous versions [5]–[7]; ii) the (sub)gradient push-method [8], [9]; iii) the dual-average method [10], [11]; and iv) distributed secondorder-based schemes [12], [13].”
Diff-DAC: Distributed Actor-Critic for Average Multitask Deep Reinforcement Learning (2017, methodology): “One of such procedures is the Hastings rule (Zhao and Sayed 2012), (Sayed 2014, p.492).”
Decentralized consensus optimization with asynchrony and delays (2016, methodology): “Under these conditions, it is shown in [33] that W has an eigenvalue 1 with multiplicity 1, and all other eigenvalues are strictly inside the unit circle.”
Robust Distributed Estimation by Networked Agents (2017, methodology): “The reader is referred to [12], [22], [23], [29] for further details on the procedure that led to the optimal combination policy (111).”
A geometrically convergent method for distributed optimization over time-varying graphs (2016, methodology): “Among all these references, methods introduced in references [11]–[15], [22], [22], [25], [26] are all primal-only-like methods that work over undirected ﬁxed graphs and usually need a diminishing step size to reach exact convergence; references [16]–[21], [23], [24], [27] concern about…”
On the influence of momentum acceleration on online learning (2016, methodology): “It is nowadays well-recognized that the small O(µ) degradation in performance is acceptable in most large-scale learning and adaptation problems (Bousquet and Bottou, 2008; Bottou, 2010; Sayed, 2014b).”