Sheaf-Based Federated Representation Learning

• URL: http://arxiv.org/abs/2608.10016v1
• Date: Sat, 08 Aug 2026 16:38:34 GMT
• Title: Sheaf-Based Federated Representation Learning
• Abstract summary: Sheaf-based Federated Representation Learning (SFRL)<n>We develop a decentralized algorithm for solving SFRL, termed Sheaf-FRL, which alternates between gradient updates of the local models and closed-form Procrustes updates of the edge-wise restriction maps.<n>As an application, we consider a cooperative classification task in the context of semantic communication.
• Score: 11.430396441085579
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Heterogeneous federated systems require agents to learn and exchange informative representations despite differences in data distributions, sensing modalities, model architectures, latent dimensionalities, and local learning objectives. To address this challenge, we propose Sheaf-based Federated Representation Learning (SFRL), a general framework that jointly optimizes local objectives with a manifold-constrained geometric alignment regularizer based on learnable sheaf restriction maps. Unlike most existing approaches, SFRL does not assume a shared global latent space. Instead, global consistency emerges from the alignment of neighboring latent representations through orthogonal transformations and isometric embeddings. This alignment is enforced by a quadratic gluing regularizer induced by the sheaf Laplacian, whose learnable restriction maps adapt the geometry to the observed data. The penalty is evaluated on a small set of shared pilot samples, ensuring scalability and communication efficiency. We develop a decentralized algorithm for solving SFRL, termed Sheaf-FRL, which alternates between gradient updates of the local models and closed-form Procrustes updates of the edge-wise restriction maps. We further establish convergence of Sheaf-FRL to first-order stationary points in both deterministic and stochastic settings. As an application, we consider a cooperative classification task in the context of semantic communication, under model and data heterogeneity. Our results show that Sheaf-FRL outperforms baseline approaches in terms of local and post-communication classification accuracy across different levels of local distribution shift and exhibits greater robustness to latent-space dimensionality compression.

Related papers

• Topology-Aware Neighborhood Learning for Source-Free Cross-Scene Hyperspectral Image Classification [11.354716471312239]
  Domain adaptation has advanced cross-scene hyperspectral image classification, significantly improving discriminative capability in complex scenarios.<n>Conventional domain adaptation methods become impractical, severely restricting their utility in realistic remote sensing scenarios.<n>We propose a topology-aware source-free learning framework to achieve robust and accurate classification without source data.
  arXiv  Detail & Related papers  (2026-08-06T12:40:06Z)
• Enhancing Gradient Inversion Attacks in Federated Learning via Hierarchical Feature Optimization [56.95448807869383]
  Federated Learning (FL) has emerged as a compelling paradigm for privacy-preserving distributed machine learning.<n>Recent studies find that the gradients exchanged in the FL system are also vulnerable to privacy leakage.<n>We propose textbfGradient textbfInversion over textbfFeature textbfDomains (GIFD)
  arXiv  Detail & Related papers  (2026-04-01T14:32:15Z)
• NERVE: Neighbourhood & Entropy-guided Random-walk for training free open-Vocabulary sEgmentation [18.627047608492795]
  We propose a training-free method for Open-Vocabulary Semantics (OVSS) called NERVE.<n>NERVE integrates global and fine-grained local information, exploiting the neighbourhood structure from the self-attention layer of a stable diffusion model.<n>Our method does not require any conventional post-processing techniques like Conditional Random Fields (CRF) or Pixel-Adaptive Mask Refinement (PAMR)
  arXiv  Detail & Related papers  (2025-11-11T13:43:57Z)
• Widening the Network Mitigates the Impact of Data Heterogeneity on FedAvg [6.185573921868495]
  Federated learning (FL) enables decentralized clients to train a model collaboratively without sharing local data.<n>We prove that the impact of data heterogeneity diminishes as the width of neural networks increases, ultimately vanishing when the width approaches infinity.<n>In the infinite-width regime, we further prove that both the global and local models in FedAvg behave as linear models, and that FedAvg achieves the same generalization performance as centralized learning with the same number of GD iterations.
  arXiv  Detail & Related papers  (2025-08-18T02:22:55Z)
• DFed-SST: Building Semantic- and Structure-aware Topologies for Decentralized Federated Graph Learning [5.072144098417946]
  Decentralized Federated Learning (DFL) has emerged as a robust distributed paradigm.<n>We propose DFed-SST, a decentralized graph learning framework with adaptive communication.<n>Experiments on eight real-world datasets consistently demonstrate the superiority of DFed-SST, achieving 3.26% improvement in average accuracy over baseline methods.
  arXiv  Detail & Related papers  (2025-08-15T15:15:54Z)
• Geometric Knowledge-Guided Localized Global Distribution Alignment for Federated Learning [7.893874342163829]
  We propose a geometry-guided data generation method that centers on simulating the global embedding distribution locally.<n>We first introduce the concept of the geometric shape of an embedding distribution.<n>We then address the challenge of obtaining global geometric shapes under privacy constraints.
  arXiv  Detail & Related papers  (2025-03-09T05:30:28Z)
• Decentralized Sporadic Federated Learning: A Unified Algorithmic Framework with Convergence Guarantees [18.24213566328972]
  Decentralized learning computation (DFL) captures FL settings where both (i) model updates and (ii) model aggregations are carried out by the clients without a central server.<n>$textttDSpodFL$, a DFL methodology built on a generalized notion of $textitsporadicity$ in both local gradient and aggregation processes.<n>$textttDSpodFL$ consistently achieves improved speeds compared with baselines under various system settings.
  arXiv  Detail & Related papers  (2024-02-05T19:02:19Z)
• Generalizable Heterogeneous Federated Cross-Correlation and Instance Similarity Learning [60.058083574671834]
  This paper presents a novel FCCL+, federated correlation and similarity learning with non-target distillation. For heterogeneous issue, we leverage irrelevant unlabeled public data for communication. For catastrophic forgetting in local updating stage, FCCL+ introduces Federated Non Target Distillation.
  arXiv  Detail & Related papers  (2023-09-28T09:32:27Z)
• Federated Learning as Variational Inference: A Scalable Expectation Propagation Approach [66.9033666087719]
  This paper extends the inference view and describes a variational inference formulation of federated learning. We apply FedEP on standard federated learning benchmarks and find that it outperforms strong baselines in terms of both convergence speed and accuracy.
  arXiv  Detail & Related papers  (2023-02-08T17:58:11Z)
• Towards Understanding and Mitigating Dimensional Collapse in Heterogeneous Federated Learning [112.69497636932955]
  Federated learning aims to train models across different clients without the sharing of data for privacy considerations. We study how data heterogeneity affects the representations of the globally aggregated models. We propose sc FedDecorr, a novel method that can effectively mitigate dimensional collapse in federated learning.
  arXiv  Detail & Related papers  (2022-10-01T09:04:17Z)
• Semi-supervised Domain Adaptive Structure Learning [72.01544419893628]
  Semi-supervised domain adaptation (SSDA) is a challenging problem requiring methods to overcome both 1) overfitting towards poorly annotated data and 2) distribution shift across domains. We introduce an adaptive structure learning method to regularize the cooperation of SSL and DA.
  arXiv  Detail & Related papers  (2021-12-12T06:11:16Z)
• Clustered Federated Learning via Generalized Total Variation Minimization [83.26141667853057]
  We study optimization methods to train local (or personalized) models for local datasets with a decentralized network structure. Our main conceptual contribution is to formulate federated learning as total variation minimization (GTV) Our main algorithmic contribution is a fully decentralized federated learning algorithm.
  arXiv  Detail & Related papers  (2021-05-26T18:07:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.