FedVSSAM: Mitigating Flatness Incompatibility in Sharpness-Aware Federated Learning

• URL: http://arxiv.org/abs/2605.09144v1
• Date: Sat, 09 May 2026 20:03:02 GMT
• Title: FedVSSAM: Mitigating Flatness Incompatibility in Sharpness-Aware Federated Learning
• Abstract summary: Sharpness-aware (SAM) is an effective method for improving the generalization of federated learning (FL) by steering local training toward flat minima.<n>We propose Federated Learning with minimization-suppressed sharpness-suppressed search (FedVSSAM)<n>FedVSSAM anchors both update directions to a more stable global direction and uses it consistently in local flatness perturbation, local update.
• Score: 65.74073318961548
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Sharpness-aware minimization (SAM) is an effective method for improving the generalization of federated learning (FL) by steering local training toward flat minima. Under data heterogeneity, however, device-side SAM searches for locally flat basins that are incompatible with the flat region preferred by the global objective. We identify this structural failure mode as flatness incompatibility, which explains why improving local flatness alone may provide limited training and generalization improvement for the global model. We reveal that flatness incompatibility arises from data heterogeneity and the friendly adversary phenomenon, and is further amplified by local updates and partial device participation. To mitigate this issue, we propose Federated Learning with variance-suppressed sharpness-aware minimization (FedVSSAM), which constructs a variance-suppressed adjusted direction and uses it consistently in local flatness search, local descent, and global update. FedVSSAM anchors both perturbation and update directions to a more stable global direction, instead of correcting only an isolated local perturbation. We establish non-convex convergence guarantees of FedVSSAM and prove that the mean-square deviation between the adjusted direction and the global gradient is effectively controlled. Experiments demonstrate that FedVSSAM mitigates flatness incompatibility and outperforms the baselines across diverse FL settings.

Related papers

• Federated Learning for Feature Generalization with Convex Constraints [9.335487623272604]
  Federated learning (FL) often struggles with generalization due to heterogeneous client data.<n>We propose FedCONST, an approach that adaptively modulates update magnitudes based on the parameter strength of the global model.
  arXiv  Detail & Related papers  (2026-06-12T12:50:33Z)
• OmniISR: A Unified Framework for Centralized and Federated Learning via Intermediate Supervision and Regularization [58.03221830946145]
  We propose OmniISR, a unified framework that fuses pure CL, pure FL, and hybrid CL-FL training modes.<n>We show that OmniISR consistently improves model performance in both centralized and federated paradigms.
  arXiv  Detail & Related papers  (2026-05-19T04:13:27Z)
• FedNSAM:Consistency of Local and Global Flatness for Federated Learning [26.41380732455181]
  We propose a novel textbfFedNSAM algorithm that accelerates the SAM algorithm by introducing global Nesterov momentum into the local update.<n>textbfFedNSAM uses the global Nesterov momentum as the direction of local estimation of client global perturbations and extrapolation.<n> Empirically, we conduct comprehensive experiments on CNN and Transformer models to verify the superior performance and efficiency of textbfFedNSAM.
  arXiv  Detail & Related papers  (2026-02-27T09:07:47Z)
• Adaptive Dual-Weighting Framework for Federated Learning via Out-of-Distribution Detection [53.45696787935487]
  Federated Learning (FL) enables collaborative model training across large-scale distributed service nodes.<n>In real-world service-oriented deployments, data generated by heterogeneous users, devices, and application scenarios are inherently non-IID.<n>We propose FLood, a novel FL framework inspired by out-of-distribution (OOD) detection.
  arXiv  Detail & Related papers  (2026-02-01T05:54:59Z)
• Beyond Local Sharpness: Communication-Efficient Global Sharpness-aware Minimization for Federated Learning [23.733942517577134]
  Federated learning (FL) enables collaborative model training with privacy preservation.<n>Recent approaches use client-side sharpness-aware minimization (SAM) to encourage flatter minima.<n>This work introduces FedGloSS, a novel FL approach that prioritizes the optimization of global sharpness on the server.
  arXiv  Detail & Related papers  (2024-12-04T22:46:06Z)
• Locally Estimated Global Perturbations are Better than Local Perturbations for Federated Sharpness-aware Minimization [81.32266996009575]
  In federated learning (FL), the multi-step update and data heterogeneity among clients often lead to a loss landscape with sharper minima. We propose FedLESAM, a novel algorithm that locally estimates the direction of global perturbation on client side.
  arXiv  Detail & Related papers  (2024-05-29T08:46:21Z)
• Stragglers-Aware Low-Latency Synchronous Federated Learning via Layer-Wise Model Updates [71.81037644563217]
  Synchronous federated learning (FL) is a popular paradigm for collaborative edge learning. As some of the devices may have limited computational resources and varying availability, FL latency is highly sensitive to stragglers. We propose straggler-aware layer-wise federated learning (SALF) that leverages the optimization procedure of NNs via backpropagation to update the global model in a layer-wise fashion.
  arXiv  Detail & Related papers  (2024-03-27T09:14:36Z)
• FedSpeed: Larger Local Interval, Less Communication Round, and Higher Generalization Accuracy [84.45004766136663]
  Federated learning is an emerging distributed machine learning framework. It suffers from the non-vanishing biases introduced by the local inconsistent optimal and the rugged client-drifts by the local over-fitting. We propose a novel and practical method, FedSpeed, to alleviate the negative impacts posed by these problems.
  arXiv  Detail & Related papers  (2023-02-21T03:55:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.