Distributed second order methods with increasing number of working nodes
- 1. University of Novi Sad Faculty of Sciences
- 2. University of Novi Sad Faculty of Technical Sciences
Description
Recently, an idling mechanism has been introduced in the context of distributed first order methods for minimization of a sum of nodes’ local convex costs over a generic, connected network. With the idling mechanism, each node i, at each iteration k, is active – updates its solution estimate and exchanges messages with its network neighborhood – with probability pk, and it stays idle with probability 1 − pk, while the activations are independent both across nodes and across iterations. In this paper, we demonstrate that the idling mechanism can be successfully incorporated in distributed second order methods also. Specifically, we apply the idling mechanism to the recently proposed Distributed Quasi Newton method (DQN). We show that, when pk grows to one across iterations geometrically, DQN with idling exhibits very similar theoretical convergence and convergence rates properties as the standard DQN method, thus achieving the same order of convergence rate (R-linear) as the standard DQN, but with significantly cheaper updates.
Files
IEEETAC2019DDNN.pdf
Files
(425.9 kB)
Name | Size | Download all |
---|---|---|
md5:39c9e71acbffdc6dd7075beb45f86fd8
|
425.9 kB | Preview Download |
Additional details
Related works
- Is supplemented by
- 10.5281/zenodo.3337787 (DOI)