On the Limit Performance of Floating Gossip

• URL: http://arxiv.org/abs/2302.08413v1
• Date: Thu, 16 Feb 2023 16:42:38 GMT
• Title: On the Limit Performance of Floating Gossip
• Authors: Gianluca Rizzo, Noelia Perez Palma, Marco Ajmone Marsan, and Vincenzo Mancuso
• Abstract summary: Gossip Learning scheme relies on Floating Content to implement location-based probabilistic evolution of machine learning models in an infrastructure-less manner. We consider dynamic scenarios where continuous learning is necessary, and we adopt a mean field approach to investigate the limit performance of Floating Gossip. Our model shows that Floating Gossip can be very effective in implementing continuous training and update of machine learning models in a cooperative manner.
• Score: 6.883143706086789
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper we investigate the limit performance of Floating Gossip, a new, fully distributed Gossip Learning scheme which relies on Floating Content to implement location-based probabilistic evolution of machine learning models in an infrastructure-less manner. We consider dynamic scenarios where continuous learning is necessary, and we adopt a mean field approach to investigate the limit performance of Floating Gossip in terms of amount of data that users can incorporate into their models, as a function of the main system parameters. Different from existing approaches in which either communication or computing aspects of Gossip Learning are analyzed and optimized, our approach accounts for the compound impact of both aspects. We validate our results through detailed simulations, proving good accuracy. Our model shows that Floating Gossip can be very effective in implementing continuous training and update of machine learning models in a cooperative manner, based on opportunistic exchanges among moving users.

Related papers

• DETAIL: Task DEmonsTration Attribution for Interpretable In-context Learning [75.68193159293425]
  In-context learning (ICL) allows transformer-based language models to learn a specific task with a few "task demonstrations" without updating their parameters. We propose an influence function-based attribution technique, DETAIL, that addresses the specific characteristics of ICL. We experimentally prove the wide applicability of DETAIL by showing our attribution scores obtained on white-box models are transferable to black-box models in improving model performance.
  arXiv  Detail & Related papers  (2024-05-22T15:52:52Z)
• TrACT: A Training Dynamics Aware Contrastive Learning Framework for Long-tail Trajectory Prediction [7.3292387742640415]
  We propose to incorporate richer training dynamics information into a prototypical contrastive learning framework. We conduct empirical evaluations of our approach using two large-scale naturalistic datasets.
  arXiv  Detail & Related papers  (2024-04-18T23:12:46Z)
• Enhancing Fairness and Performance in Machine Learning Models: A Multi-Task Learning Approach with Monte-Carlo Dropout and Pareto Optimality [1.5498930424110338]
  This study introduces an approach to mitigate bias in machine learning by leveraging model uncertainty. Our approach utilizes a multi-task learning (MTL) framework combined with Monte Carlo (MC) Dropout to assess and mitigate uncertainty in predictions related to protected labels.
  arXiv  Detail & Related papers  (2024-04-12T04:17:50Z)
• Federated Learning with Projected Trajectory Regularization [65.6266768678291]
  Federated learning enables joint training of machine learning models from distributed clients without sharing their local data. One key challenge in federated learning is to handle non-identically distributed data across the clients. We propose a novel federated learning framework with projected trajectory regularization (FedPTR) for tackling the data issue.
  arXiv  Detail & Related papers  (2023-12-22T02:12:08Z)
• ALP: Action-Aware Embodied Learning for Perception [60.64801970249279]
  We introduce Action-Aware Embodied Learning for Perception (ALP) ALP incorporates action information into representation learning through a combination of optimizing a reinforcement learning policy and an inverse dynamics prediction objective. We show that ALP outperforms existing baselines in several downstream perception tasks.
  arXiv  Detail & Related papers  (2023-06-16T21:51:04Z)
• Towards Compute-Optimal Transfer Learning [82.88829463290041]
  We argue that zero-shot structured pruning of pretrained models allows them to increase compute efficiency with minimal reduction in performance. Our results show that pruning convolutional filters of pretrained models can lead to more than 20% performance improvement in low computational regimes.
  arXiv  Detail & Related papers  (2023-04-25T21:49:09Z)
• Batch Active Learning from the Perspective of Sparse Approximation [12.51958241746014]
  Active learning enables efficient model training by leveraging interactions between machine learning agents and human annotators. We study and propose a novel framework that formulates batch active learning from the sparse approximation's perspective. Our active learning method aims to find an informative subset from the unlabeled data pool such that the corresponding training loss function approximates its full data pool counterpart.
  arXiv  Detail & Related papers  (2022-11-01T03:20:28Z)
• Learning by Distillation: A Self-Supervised Learning Framework for Optical Flow Estimation [71.76008290101214]
  DistillFlow is a knowledge distillation approach to learning optical flow. It achieves state-of-the-art unsupervised learning performance on both KITTI and Sintel datasets. Our models ranked 1st among all monocular methods on the KITTI 2015 benchmark, and outperform all published methods on the Sintel Final benchmark.
  arXiv  Detail & Related papers  (2021-06-08T09:13:34Z)
• A Variational Infinite Mixture for Probabilistic Inverse Dynamics Learning [34.90240171916858]
  We develop an efficient variational Bayes inference technique for infinite mixtures of probabilistic local models. We highlight the model's power in combining data-driven adaptation, fast prediction and the ability to deal with discontinuous functions and heteroscedastic noise. We use the learned models for online dynamics control of a Barrett-WAM manipulator, significantly improving the trajectory tracking performance.
  arXiv  Detail & Related papers  (2020-11-10T16:15:13Z)
• Learning Diverse Representations for Fast Adaptation to Distribution Shift [78.83747601814669]
  We present a method for learning multiple models, incorporating an objective that pressures each to learn a distinct way to solve the task. We demonstrate our framework's ability to facilitate rapid adaptation to distribution shift.
  arXiv  Detail & Related papers  (2020-06-12T12:23:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.