Personalized Federated Learning for Spatio-Temporal Forecasting: A Dual Semantic Alignment-Based Contrastive Approach

• URL: http://arxiv.org/abs/2404.03702v1
• Date: Thu, 4 Apr 2024 02:43:56 GMT
• Title: Personalized Federated Learning for Spatio-Temporal Forecasting: A Dual Semantic Alignment-Based Contrastive Approach
• Abstract summary: Federated dUal paradigm, sEtic aLignment-based FUELStive learning, proposed. Experiments demonstrate that FUELS outperforms state-of-the-art methods, with communication cost decreasing by around 94%.
• Score: 14.145373808547122
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The existing federated learning (FL) methods for spatio-temporal forecasting fail to capture the inherent spatio-temporal heterogeneity, which calls for personalized FL (PFL) methods to model the spatio-temporally variant patterns. While contrastive learning approach is promising in addressing spatio-temporal heterogeneity, the existing methods are noneffective in determining negative pairs and can hardly apply to PFL paradigm. To tackle this limitation, we propose a novel PFL method, named Federated dUal sEmantic aLignment-based contraStive learning (FUELS), which can adaptively align positive and negative pairs based on semantic similarity, thereby injecting precise spatio-temporal heterogeneity into the latent representation space by auxiliary contrastive tasks. From temporal perspective, a hard negative filtering module is introduced to dynamically align heterogeneous temporal representations for the supplemented intra-client contrastive task. From spatial perspective, we design lightweight-but-efficient prototypes as client-level semantic representations, based on which the server evaluates spatial similarity and yields client-customized global prototypes for the supplemented inter-client contrastive task. Extensive experiments demonstrate that FUELS outperforms state-of-the-art methods, with communication cost decreasing by around 94%.

Related papers

• Decoupled Contrastive Learning for Federated Learning [0.0]
  We introduce Decoupled Contrastive Learning for Federated Learning (DCFL)<n>DCFL is a novel framework that decouples the existing contrastive loss into two objectives.<n>Our results show that DCFL achieves stronger alignment between positive samples and greater uniformity between negative samples.
  arXiv  Detail & Related papers  (2025-08-06T01:39:54Z)
• A theoretical framework for self-supervised contrastive learning for continuous dependent data [86.50780641055258]
  Self-supervised learning (SSL) has emerged as a powerful approach to learning representations, particularly in the field of computer vision.<n>We propose a novel theoretical framework for contrastive SSL tailored to emphsemantic independence between samples.<n>Specifically, we outperform TS2Vec on the standard UEA and UCR benchmarks, with accuracy improvements of $4.17$% and $2.08$%, respectively.
  arXiv  Detail & Related papers  (2025-06-11T14:23:47Z)
• Adaptive Deadline and Batch Layered Synchronized Federated Learning [66.93447103966439]
  Federated learning (FL) enables collaborative model training across distributed edge devices while preserving data privacy, and typically operates in a round-based synchronous manner.<n>We propose ADEL-FL, a novel framework that jointly optimize per-round deadlines and user-specific batch sizes for layer-wise aggregation.
  arXiv  Detail & Related papers  (2025-05-29T19:59:18Z)
• DiffPO: Diffusion-styled Preference Optimization for Efficient Inference-Time Alignment of Large Language Models [50.32663816994459]
  Diffusion-styled Preference Optimization (model) provides an efficient and policy-agnostic solution for aligning LLMs with humans. modelavoids the time latency associated with token-level generation. Experiments on AlpacaEval 2, MT-bench, and HH-RLHF demonstrate that modelachieves superior alignment performance across various settings.
  arXiv  Detail & Related papers  (2025-03-06T09:21:54Z)
• FedSA: A Unified Representation Learning via Semantic Anchors for Prototype-based Federated Learning [4.244188591221394]
  We propose a novel framework named Federated Learning via Semantic Anchors (FedSA) to decouple the generation of prototypes from local representation learning. FedSA significantly outperforms existing prototype-based FL methods on various classification tasks.
  arXiv  Detail & Related papers  (2025-01-09T16:10:03Z)
• Cross Space and Time: A Spatio-Temporal Unitized Model for Traffic Flow Forecasting [16.782154479264126]
  Predicting backbone-temporal traffic flow presents challenges due to complex interactions between temporal factors. Existing approaches address these dimensions in isolation, neglecting their critical interdependencies. In this paper, we introduce Sanonymous-Temporal Unitized Unitized Cell (ASTUC), a unified framework designed to capture both spatial and temporal dependencies.
  arXiv  Detail & Related papers  (2024-11-14T07:34:31Z)
• Learning Discriminative Spatio-temporal Representations for Semi-supervised Action Recognition [23.44320273156057]
  We propose an Adaptive Contrastive Learning(ACL) strategy and a Multi-scale Temporal Learning(MTL) strategy. ACL strategy assesses the confidence of all unlabeled samples by the class prototypes of the labeled data, and adaptively selects positive-negative samples from a pseudo-labeled sample bank to construct contrastive learning. MTL strategy could highlight informative semantics from long-term clips and integrate them into the short-term clip while suppressing noisy information.
  arXiv  Detail & Related papers  (2024-04-25T08:49:08Z)
• Interactive Test-Time Adaptation with Reliable Spatial-Temporal Voxels for Multi-Modal Segmentation [56.70910056845503]
  Multi-modal test-time adaptation (MM-TTA) adapts models to an unlabeled target domain by leveraging the complementary multi-modal inputs in an online manner.<n>Previous MM-TTA methods for 3D segmentation suffer from two major limitations: 1) unstable frame-wise predictions caused by temporal inconsistency, and 2) consistently incorrect predictions that violate the assumption of reliable modality guidance.<n>This work introduces a comprehensive two-fold framework: Latte++ that better suppresses the unstable frame-wise predictions with more informative geometric correspondences, and Interactive Test-Time Adaptation (ITTA), a flexible add-on to empower effortless human feedback
  arXiv  Detail & Related papers  (2024-03-11T06:56:08Z)
• FedAA: A Reinforcement Learning Perspective on Adaptive Aggregation for Fair and Robust Federated Learning [5.622065847054885]
  Federated Learning (FL) has emerged as a promising approach for privacy-preserving model training across decentralized devices. We introduce a novel method called textbfFedAA, which optimize client contributions via textbfAdaptive textbfAggregation to enhance model robustness against malicious clients.
  arXiv  Detail & Related papers  (2024-02-08T10:22:12Z)
• Prototypical Contrastive Learning through Alignment and Uniformity for Recommendation [6.790779112538357]
  We present underlinePrototypical contrastive learning through underlineAlignment and underlineUniformity for recommendation. Specifically, we first propose prototypes as a latent space to ensure consistency across different augmentations from the origin graph. The absence of explicit negatives means that directly optimizing the consistency loss between instance and prototype could easily result in dimensional collapse issues.
  arXiv  Detail & Related papers  (2024-02-03T08:19:26Z)
• Time-series Generation by Contrastive Imitation [87.51882102248395]
  We study a generative framework that seeks to combine the strengths of both: Motivated by a moment-matching objective to mitigate compounding error, we optimize a local (but forward-looking) transition policy. At inference, the learned policy serves as the generator for iterative sampling, and the learned energy serves as a trajectory-level measure for evaluating sample quality.
  arXiv  Detail & Related papers  (2023-11-02T16:45:25Z)
• Revisiting the Temporal Modeling in Spatio-Temporal Predictive Learning under A Unified View [73.73667848619343]
  We introduce USTEP (Unified S-TEmporal Predictive learning), an innovative framework that reconciles the recurrent-based and recurrent-free methods by integrating both micro-temporal and macro-temporal scales.
  arXiv  Detail & Related papers  (2023-10-09T16:17:42Z)
• Towards Instance-adaptive Inference for Federated Learning [80.38701896056828]
  Federated learning (FL) is a distributed learning paradigm that enables multiple clients to learn a powerful global model by aggregating local training. In this paper, we present a novel FL algorithm, i.e., FedIns, to handle intra-client data heterogeneity by enabling instance-adaptive inference in the FL framework. Our experiments show that our FedIns outperforms state-of-the-art FL algorithms, e.g., a 6.64% improvement against the top-performing method with less than 15% communication cost on Tiny-ImageNet.
  arXiv  Detail & Related papers  (2023-08-11T09:58:47Z)
• STS-CCL: Spatial-Temporal Synchronous Contextual Contrastive Learning for Urban Traffic Forecasting [4.947443433688782]
  This work employs the advanced contrastive learning and proposes a novel Spatial-Temporalous Contextual Contrastive Learning (STS-CCL) model. Experiments and evaluations demonstrate that building a predictor upon STS-CCL contrastive learning model gains superior performance than existing traffic forecasting benchmarks.
  arXiv  Detail & Related papers  (2023-07-05T03:47:28Z)
• Spatio-temporal Diffusion Point Processes [23.74522530140201]
  patio-temporal point process (STPP) is a collection of events accompanied with time and space. The failure to model the joint distribution leads to limited capacities in characterizing the pasthua-temporal interactions given events. We propose a novel parameterization framework, which learns complex spatial-temporal joint distributions. Our framework outperforms the state-of-the-art baselines remarkably, with an average improvement over 50%.
  arXiv  Detail & Related papers  (2023-05-21T08:53:00Z)
• Beyond Instance Discrimination: Relation-aware Contrastive Self-supervised Learning [75.46664770669949]
  We present relation-aware contrastive self-supervised learning (ReCo) to integrate instance relations. Our ReCo consistently gains remarkable performance improvements.
  arXiv  Detail & Related papers  (2022-11-02T03:25:28Z)
• DEALIO: Data-Efficient Adversarial Learning for Imitation from Observation [57.358212277226315]
  In imitation learning from observation IfO, a learning agent seeks to imitate a demonstrating agent using only observations of the demonstrated behavior without access to the control signals generated by the demonstrator. Recent methods based on adversarial imitation learning have led to state-of-the-art performance on IfO problems, but they typically suffer from high sample complexity due to a reliance on data-inefficient, model-free reinforcement learning algorithms. This issue makes them impractical to deploy in real-world settings, where gathering samples can incur high costs in terms of time, energy, and risk. We propose a more data-efficient IfO algorithm
  arXiv  Detail & Related papers  (2021-03-31T23:46:32Z)
• Unleashing the Power of Contrastive Self-Supervised Visual Models via Contrast-Regularized Fine-Tuning [94.35586521144117]
  We investigate whether applying contrastive learning to fine-tuning would bring further benefits. We propose Contrast-regularized tuning (Core-tuning), a novel approach for fine-tuning contrastive self-supervised visual models.
  arXiv  Detail & Related papers  (2021-02-12T16:31:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.