On Gossip Algorithms for Machine Learning with Pairwise Objectives

• URL: http://arxiv.org/abs/2603.24128v1
• Date: Wed, 25 Mar 2026 09:43:17 GMT
• Title: On Gossip Algorithms for Machine Learning with Pairwise Objectives
• Abstract summary: In the IoT era, information is more and more frequently picked up by connected smart sensors with increasing, though limited, storage, communication and computation abilities.<n>The development of statistical learning methods dedicated to data that are shared over a network is now a major issue.<n>Gossip-based algorithms have been developed for the purpose of solving a wide variety of statistical learning tasks.
• Score: 17.859587616935002
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In the IoT era, information is more and more frequently picked up by connected smart sensors with increasing, though limited, storage, communication and computation abilities. Whether due to privacy constraints or to the structure of the distributed system, the development of statistical learning methods dedicated to data that are shared over a network is now a major issue. Gossip-based algorithms have been developed for the purpose of solving a wide variety of statistical learning tasks, ranging from data aggregation over sensor networks to decentralized multi-agent optimization. Whereas the vast majority of contributions consider situations where the function to be estimated or optimized is a basic average of individual observations, it is the goal of this article to investigate the case where the latter is of pairwise nature, taking the form of a U -statistic of degree two. Motivated by various problems such as similarity learning, ranking or clustering for instance, we revisit gossip algorithms specifically designed for pairwise objective functions and provide a comprehensive theoretical framework for their convergence. This analysis fills a gap in the literature by establishing conditions under which these methods succeed, and by identifying the graph properties that critically affect their efficiency. In particular, a refined analysis of the convergence upper and lower bounds is performed.

Related papers

• Beyond Centralization: Provable Communication Efficient Decentralized Multi-Task Learning [7.147459905796264]
  We study decentralized multi-task representation learning in which the features share a low-rank structure.<n>In the decentralized setting, task data are distributed across multiple nodes, and information exchange between nodes is constrained by a communication network.
  arXiv  Detail & Related papers  (2025-12-27T18:44:43Z)
• Similarity-based fuzzy clustering scientific articles: potentials and challenges from mathematical and computational perspectives [0.0]
  Fuzzy clustering plays a vital role in analyzing publication data.<n>This problem can be formulated as a constrained optimization model, where the goal is to minimize the discrepancy between the similarity observed from data and the similarity derived from a predicted distribution.<n>We analyze potentials and challenges of the approach from both mathematical and computational perspectives.
  arXiv  Detail & Related papers  (2025-06-04T15:10:31Z)
• Optimization and Learning in Open Multi-Agent Systems [1.0249620437941]
  This article introduces a novel distributed algorithm to address a broad class of problems in "open networks"<n>The proposed algorithm is used to solve dynamic consensus or tracking problems on different metrics of interest.
  arXiv  Detail & Related papers  (2025-01-28T10:40:09Z)
• Distributed Learning over Networks with Graph-Attention-Based Personalization [49.90052709285814]
  We propose a graph-based personalized algorithm (GATTA) for distributed deep learning. In particular, the personalized model in each agent is composed of a global part and a node-specific part. By treating each agent as one node in a graph the node-specific parameters as its features, the benefits of the graph attention mechanism can be inherited.
  arXiv  Detail & Related papers  (2023-05-22T13:48:30Z)
• Contextual Model Aggregation for Fast and Robust Federated Learning in Edge Computing [88.76112371510999]
  Federated learning is a prime candidate for distributed machine learning at the network edge. Existing algorithms face issues with slow convergence and/or robustness of performance. We propose a contextual aggregation scheme that achieves the optimal context-dependent bound on loss reduction.
  arXiv  Detail & Related papers  (2022-03-23T21:42:31Z)
• Dif-MAML: Decentralized Multi-Agent Meta-Learning [54.39661018886268]
  We propose a cooperative multi-agent meta-learning algorithm, referred to as MAML or Dif-MAML. We show that the proposed strategy allows a collection of agents to attain agreement at a linear rate and to converge to a stationary point of the aggregate MAML. Simulation results illustrate the theoretical findings and the superior performance relative to the traditional non-cooperative setting.
  arXiv  Detail & Related papers  (2020-10-06T16:51:09Z)
• Learning while Respecting Privacy and Robustness to Distributional Uncertainties and Adversarial Data [66.78671826743884]
  The distributionally robust optimization framework is considered for training a parametric model. The objective is to endow the trained model with robustness against adversarially manipulated input data. Proposed algorithms offer robustness with little overhead.
  arXiv  Detail & Related papers  (2020-07-07T18:25:25Z)
• Scaling-up Distributed Processing of Data Streams for Machine Learning [10.581140430698103]
  This paper reviews recently developed methods that focus on large-scale distributed optimization in the compute- and bandwidth-limited regime. It focuses on methods that solve: (i) distributed convex problems, and (ii) distributed principal component analysis, which is a non problem with geometric structure that permits global convergence.
  arXiv  Detail & Related papers  (2020-05-18T16:28:54Z)
• Dynamic Federated Learning [57.14673504239551]
  Federated learning has emerged as an umbrella term for centralized coordination strategies in multi-agent environments. We consider a federated learning model where at every iteration, a random subset of available agents perform local updates based on their data. Under a non-stationary random walk model on the true minimizer for the aggregate optimization problem, we establish that the performance of the architecture is determined by three factors, namely, the data variability at each agent, the model variability across all agents, and a tracking term that is inversely proportional to the learning rate of the algorithm.
  arXiv  Detail & Related papers  (2020-02-20T15:00:54Z)
• Distributed Learning in the Non-Convex World: From Batch to Streaming Data, and Beyond [73.03743482037378]
  Distributed learning has become a critical direction of the massively connected world envisioned by many. This article discusses four key elements of scalable distributed processing and real-time data computation problems. Practical issues and future research will also be discussed.
  arXiv  Detail & Related papers  (2020-01-14T14:11:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.