Decentralized Stochastic Subgradient-type Methods with Communication Compression for Nonsmooth Nonconvex Optimization

• URL: http://arxiv.org/abs/2607.01755v1
• Date: Thu, 02 Jul 2026 06:11:38 GMT
• Title: Decentralized Stochastic Subgradient-type Methods with Communication Compression for Nonsmooth Nonconvex Optimization
• Abstract summary: In this paper, we consider the nonsmooth non decentralized optimization problem, where inter-agent communication is compressed.<n>We propose a general framework that unifies various decentralized subgradient-type methods with unbiased compression and contractive compression with error compensation.
• Score: 14.561370824112524
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we consider the nonsmooth nonconvex decentralized optimization problem, where inter-agent communication is compressed. We propose a general framework that unifies various decentralized stochastic subgradient-type methods with unbiased compression and contractive compression with error compensation. By relating the consensus-error iterates and the averaged iterates to the trajectories of continuous-time differential inclusions, we establish global convergence for all methods encompassed by our framework when the objective functions are nonsmooth and lack Clarke regularity. Based on our framework, we further develop several compression-based methods, including decentralized stochastic subgradient methods utilizing sign-based regularization and gradient-tracking momentum. Preliminary numerical experiments empirically support our theoretical results and highlight the communication-accuracy trade-off of the newly developed methods.

Related papers

• Stability and Generalization of Push-Sum Based Decentralized Optimization over Directed Graphs [55.77845440440496]
  Push-based decentralized communication enables optimization over communication networks, where information exchange may be asymmetric.<n>We develop a unified uniform-stability framework for the Gradient Push (SGP) algorithm.<n>A key technical ingredient is an imbalance-aware generalization bound through two quantities.
  arXiv  Detail & Related papers  (2026-02-24T05:32:03Z)
• Compressed Decentralized Momentum Stochastic Gradient Methods for Nonconvex Optimization [20.588374906635256]
  We design two compressed decentralized algorithms for solving non-compression heterogeneity optimization under two different scenarios.<n>Both algorithms adopt a momentum technique to achieve fast convergence and a message-speed technique to save communication costs.<n> Superior empirical performance is observed over state-of-the-art methods on deep neural networks (DNNs) and Transformers.
  arXiv  Detail & Related papers  (2025-08-07T00:33:00Z)
• A Unified Theory of Stochastic Proximal Point Methods without Smoothness [52.30944052987393]
  Proximal point methods have attracted considerable interest owing to their numerical stability and robustness against imperfect tuning. This paper presents a comprehensive analysis of a broad range of variations of the proximal point method (SPPM)
  arXiv  Detail & Related papers  (2024-05-24T21:09:19Z)
• Flattened one-bit stochastic gradient descent: compressed distributed optimization with controlled variance [55.01966743652196]
  We propose a novel algorithm for distributed gradient descent (SGD) with compressed gradient communication in the parameter-server framework. Our gradient compression technique, named flattened one-bit gradient descent (FO-SGD), relies on two simple algorithmic ideas.
  arXiv  Detail & Related papers  (2024-05-17T21:17:27Z)
• Convergence of Decentralized Stochastic Subgradient-based Methods for Nonsmooth Nonconvex functions [10.278310909980576]
  We propose a general framework that unifies various decentralized subgradient-based methods.<n>We prove convergence guarantees for some well-recognized decentralized subgradient-based methods.
  arXiv  Detail & Related papers  (2024-03-18T08:35:17Z)
• On the Communication Complexity of Decentralized Bilevel Optimization [40.45379954138305]
  We propose two novel decentralized bilevel gradient descent algorithms based on simultaneous and alternating update strategies. Our algorithms can achieve faster convergence rates and lower communication costs than existing methods. This is the first time such favorable theoretical results have been achieved with mild assumptions in the heterogeneous setting.
  arXiv  Detail & Related papers  (2023-11-19T14:56:26Z)
• On Arbitrary Compression for Decentralized Consensus and Stochastic Optimization over Directed Networks [0.6526824510982799]
  We propose an iterative-based algorithm that compresses messages according to a desired compression ratio. Contrary to existing literature allow arbitrary compression ratios. We show explicit convergence rates for decentralized optimization problems on smooth functions.
  arXiv  Detail & Related papers  (2022-04-18T04:41:56Z)
• Distributed and Stochastic Optimization Methods with Gradient Compression and Local Steps [0.0]
  We propose theoretical frameworks for the analysis and distributed methods with error compensation and local updates. We develop more than 20 new optimization methods, including the first linearly converging Error-pensated and first distributed Local-SGD methods.
  arXiv  Detail & Related papers  (2021-12-20T16:12:54Z)
• Optimization on manifolds: A symplectic approach [127.54402681305629]
  We propose a dissipative extension of Dirac's theory of constrained Hamiltonian systems as a general framework for solving optimization problems. Our class of (accelerated) algorithms are not only simple and efficient but also applicable to a broad range of contexts.
  arXiv  Detail & Related papers  (2021-07-23T13:43:34Z)
• A Linearly Convergent Algorithm for Decentralized Optimization: Sending Less Bits for Free! [72.31332210635524]
  Decentralized optimization methods enable on-device training of machine learning models without a central coordinator. We propose a new randomized first-order method which tackles the communication bottleneck by applying randomized compression operators. We prove that our method can solve the problems without any increase in the number of communications compared to the baseline.
  arXiv  Detail & Related papers  (2020-11-03T13:35:53Z)
• Linear Convergent Decentralized Optimization with Compression [50.44269451541387]
  Existing decentralized algorithms with compression mainly focus on compressing DGD-type algorithms. Motivated by primal-dual algorithms, this paper proposes first underlineLinunderlineEAr convergent. underlineDecentralized with compression, LEAD.
  arXiv  Detail & Related papers  (2020-07-01T04:35:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.