FedEMA: Federated Exponential Moving Averaging with Negative Entropy Regularizer in Autonomous Driving

• URL: http://arxiv.org/abs/2505.00318v1
• Date: Thu, 01 May 2025 05:37:43 GMT
• Title: FedEMA: Federated Exponential Moving Averaging with Negative Entropy Regularizer in Autonomous Driving
• Authors: Wei-Bin Kou, Guangxu Zhu, Bingyang Cheng, Shuai Wang, Ming Tang, Yik-Chung Wu,
• Abstract summary: Street Scene Semantic Understanding (denoted as S3U) is a crucial but complex task for autonomous driving (AD) vehicles.<n>Their inference models typically face poor generalization due to domain-shift.<n>This paper proposes Federated Exponential Moving Average (FedEMA), a novel framework that addresses this challenge through two integral innovations.
• Score: 28.013875789806725
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Street Scene Semantic Understanding (denoted as S3U) is a crucial but complex task for autonomous driving (AD) vehicles. Their inference models typically face poor generalization due to domain-shift. Federated Learning (FL) has emerged as a promising paradigm for enhancing the generalization of AD models through privacy-preserving distributed learning. However, these FL AD models face significant temporal catastrophic forgetting when deployed in dynamically evolving environments, where continuous adaptation causes abrupt erosion of historical knowledge. This paper proposes Federated Exponential Moving Average (FedEMA), a novel framework that addresses this challenge through two integral innovations: (I) Server-side model's historical fitting capability preservation via fusing current FL round's aggregation model and a proposed previous FL round's exponential moving average (EMA) model; (II) Vehicle-side negative entropy regularization to prevent FL models' possible overfitting to EMA-introduced temporal patterns. Above two strategies empower FedEMA a dual-objective optimization that balances model generalization and adaptability. In addition, we conduct theoretical convergence analysis for the proposed FedEMA. Extensive experiments both on Cityscapes dataset and Camvid dataset demonstrate FedEMA's superiority over existing approaches, showing 7.12% higher mean Intersection-over-Union (mIoU).

Related papers

• Towards Performance-Enhanced Model-Contrastive Federated Learning using Historical Information in Heterogeneous Scenarios [13.567036484228344]
  Federated Learning (FL) enables multiple nodes to collaboratively train a model without sharing raw data.<n>This paper proposes PMFL, a performance-enhanced model-contrastive federated learning framework using historical training information.
  arXiv  Detail & Related papers  (2026-02-12T13:40:37Z)
• FedDSR: Federated Deep Supervision and Regularization Towards Autonomous Driving [32.600054594223096]
  Federated Deep Supervision and Regularization (FedDSR) is a paradigm that incorporates multi-access intermediate layer supervision and regularization within federated AD system.<n>FedDSR achieves up to 8.93% improvement in mIoU and 28.57% reduction in training rounds, compared to other Federated Learning baselines.
  arXiv  Detail & Related papers  (2025-12-07T06:23:59Z)
• Which Layer Causes Distribution Deviation? Entropy-Guided Adaptive Pruning for Diffusion and Flow Models [77.55829017952728]
  EntPruner is an entropy-guided automatic progressive pruning framework for diffusion and flow models.<n>Experiments on DiT and SiT models demonstrate the effectiveness of EntPruner, achieving up to 2.22$times$ inference speedup.
  arXiv  Detail & Related papers  (2025-11-26T07:20:48Z)
• FedWCM: Unleashing the Potential of Momentum-based Federated Learning in Long-Tailed Scenarios [14.18492489954482]
  Federated Learning (FL) enables decentralized model training while preserving data privacy.<n>Despite its benefits, FL faces challenges with non-identically distributed (non-IID) data.<n>We propose FedWCM, a method that dynamically adjusts momentum using global and per-round data.
  arXiv  Detail & Related papers  (2025-07-20T14:24:57Z)
• Efficient Federated Learning with Timely Update Dissemination [54.668309196009204]
  Federated Learning (FL) has emerged as a compelling methodology for the management of distributed data.<n>We propose an efficient FL approach that capitalizes on additional downlink bandwidth resources to ensure timely update dissemination.
  arXiv  Detail & Related papers  (2025-07-08T14:34:32Z)
• Elucidated Rolling Diffusion Models for Probabilistic Weather Forecasting [52.6508222408558]
  We introduce Elucidated Rolling Diffusion Models (ERDM)<n>ERDM is the first framework to unify a rolling forecast structure with the principled, performant design of Elucidated Diffusion Models (EDM)<n>On 2D Navier-Stokes simulations and ERA5 global weather forecasting at 1.5circ resolution, ERDM consistently outperforms key diffusion-based baselines.
  arXiv  Detail & Related papers  (2025-06-24T21:44:31Z)
• FedSKD: Aggregation-free Model-heterogeneous Federated Learning using Multi-dimensional Similarity Knowledge Distillation [7.944298319589845]
  Federated learning (FL) enables privacy-preserving collaborative model training without direct data sharing. Model-heterogeneous FL (MHFL) allows clients to train personalized models with heterogeneous architectures tailored to their computational resources and application-specific needs. While peer-to-peer (P2P) FL removes server dependence, it suffers from model drift and knowledge dilution, limiting its effectiveness in heterogeneous settings. We propose FedSKD, a novel MHFL framework that facilitates direct knowledge exchange through round-robin model circulation.
  arXiv  Detail & Related papers  (2025-03-23T05:33:10Z)
• Biased Federated Learning under Wireless Heterogeneity [7.3716675761469945]
  Federated learning (FL) is a promising framework for computation, enabling collaborative model training without sharing private data.<n>Existing wireless computation works primarily adopt two communication strategies: (1) over-the-air (OTA) which exploits wireless signal superposition, and (2) over-the-air (OTA) which allocates resources for convergence.<n>This paper proposes novel OTA and digital FL updates that allow a structured, time-in-place bias, thereby reducing variance in FL updates.
  arXiv  Detail & Related papers  (2025-03-08T05:55:14Z)
• SEAFL: Enhancing Efficiency in Semi-Asynchronous Federated Learning through Adaptive Aggregation and Selective Training [26.478852701376294]
  We present em SEAFL, a novel FL framework designed to mitigate both the straggler and the stale model challenges in semi-asynchronous FL.<n>em SEAFL dynamically assigns weights to uploaded models during aggregation based on their staleness and importance to the current global model.<n>We evaluate the effectiveness of em SEAFL through extensive experiments on three benchmark datasets.
  arXiv  Detail & Related papers  (2025-02-22T05:13:53Z)
• MITA: Bridging the Gap between Model and Data for Test-time Adaptation [68.62509948690698]
  Test-Time Adaptation (TTA) has emerged as a promising paradigm for enhancing the generalizability of models. We propose Meet-In-The-Middle based MITA, which introduces energy-based optimization to encourage mutual adaptation of the model and data from opposing directions.
  arXiv  Detail & Related papers  (2024-10-12T07:02:33Z)
• Model Inversion Attacks Through Target-Specific Conditional Diffusion Models [54.69008212790426]
  Model inversion attacks (MIAs) aim to reconstruct private images from a target classifier's training set, thereby raising privacy concerns in AI applications. Previous GAN-based MIAs tend to suffer from inferior generative fidelity due to GAN's inherent flaws and biased optimization within latent space. We propose Diffusion-based Model Inversion (Diff-MI) attacks to alleviate these issues.
  arXiv  Detail & Related papers  (2024-07-16T06:38:49Z)
• Stragglers-Aware Low-Latency Synchronous Federated Learning via Layer-Wise Model Updates [71.81037644563217]
  Synchronous federated learning (FL) is a popular paradigm for collaborative edge learning. As some of the devices may have limited computational resources and varying availability, FL latency is highly sensitive to stragglers. We propose straggler-aware layer-wise federated learning (SALF) that leverages the optimization procedure of NNs via backpropagation to update the global model in a layer-wise fashion.
  arXiv  Detail & Related papers  (2024-03-27T09:14:36Z)
• Knowledge Rumination for Client Utility Evaluation in Heterogeneous Federated Learning [12.50871784200551]
  Federated Learning (FL) allows several clients to cooperatively train machine learning models without disclosing the raw data.<n>Non-IID data and stale models pose significant challenges to AFL, as they can diminish the practicality of the global model and even lead to training failures.<n>We propose a novel AFL framework called Federated Historical Learning (FedHist), which effectively addresses the challenges posed by both Non-IID data and gradient staleness.
  arXiv  Detail & Related papers  (2023-12-16T11:40:49Z)
• EvoFed: Leveraging Evolutionary Strategies for Communication-Efficient Federated Learning [15.124439914522693]
  Federated Learning (FL) is a decentralized machine learning paradigm that enables collaborative model training across dispersed nodes. This paper presents EvoFed, a novel approach that integrates Evolutionary Strategies (ES) with FL to address these challenges.
  arXiv  Detail & Related papers  (2023-11-13T17:25:06Z)
• Super-model ecosystem: A domain-adaptation perspective [101.76769818069072]
  This paper attempts to establish the theoretical foundation for the emerging super-model paradigm via domain adaptation. Super-model paradigms help reduce computational and data cost and carbon emission, which is critical to AI industry.
  arXiv  Detail & Related papers  (2022-08-30T09:09:43Z)
• Regularizing Variational Autoencoder with Diversity and Uncertainty Awareness [61.827054365139645]
  Variational Autoencoder (VAE) approximates the posterior of latent variables based on amortized variational inference. We propose an alternative model, DU-VAE, for learning a more Diverse and less Uncertain latent space.
  arXiv  Detail & Related papers  (2021-10-24T07:58:13Z)
• Gradual Federated Learning with Simulated Annealing [26.956032164461377]
  Federated averaging (FedAvg) is a popular federated learning (FL) technique that updates the global model by averaging local models. In this paper, we propose a new FL technique based on simulated annealing. We show that SAFL outperforms the conventional FedAvg technique in terms of the convergence speed and the classification accuracy.
  arXiv  Detail & Related papers  (2021-10-11T11:57:56Z)
• Discrete Auto-regressive Variational Attention Models for Text Modeling [53.38382932162732]
  Variational autoencoders (VAEs) have been widely applied for text modeling. They are troubled by two challenges: information underrepresentation and posterior collapse. We propose Discrete Auto-regressive Variational Attention Model (DAVAM) to address the challenges.
  arXiv  Detail & Related papers  (2021-06-16T06:36:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.