From Coordinate Matching to Structural Alignment: Rethinking Prototype Alignment in Heterogeneous Federated Learning

• URL: http://arxiv.org/abs/2605.05959v1
• Date: Thu, 07 May 2026 10:06:26 GMT
• Title: From Coordinate Matching to Structural Alignment: Rethinking Prototype Alignment in Heterogeneous Federated Learning
• Abstract summary: Heterogeneous federated learning (HtFL) aims to enable collaboration among clients that differ in both data distributions and model architectures.<n>Existing prototype-based HtFL methods typically reuse the MSE-based or cosine-based alignment mechanism.<n>We propose FedSAF, which shifts the alignment objective from absolute coordinates to inter-class relational structure.
• Score: 23.704003173950905
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Heterogeneous federated learning (HtFL) aims to enable collaboration among clients that differ in both data distributions and model architectures. Prototype-based methods, which communicate class-level feature centers (prototypes) instead of full model parameters, have recently shown strong potential for HtFL. Existing prototype-based HtFL methods typically reuse the MSE-based or cosine-based alignment mechanism developed for homogeneous FL when aligning client-specific representations with global prototypes. These approaches are essentially coordinate alignment, where representations of clients are forced to match the global prototypes in the embedding space in an element-wise manner. Such alignment implicitly assumes that all clients should map their representations into the feature subspace defined by the global prototypes. This assumption is reasonable in homogeneous FL, where all clients share the same feature extractor. However, it becomes problematic in HtFL, since heterogeneous feature extractors naturally induce client-specific feature subspaces, and forcing all clients to optimize within a single global subspace unnecessarily suppresses their learning capacity. We observe that coordinate alignment implicitly couples two distinct objectives: aligning inter-class semantic structure, which is directly beneficial for classification, and enforcing a shared feature basis, which is unnecessary and even harmful under model heterogeneity. Building on this insight, we design FedSAF, which shifts the alignment objective from absolute coordinates to inter-class relational structure. We demonstrate that structural alignment consistently outperforms coordinate alignment in heterogeneous settings. Experiments on multiple benchmarks show that our structural alignment outperforms state-of-the-art prototype-based HtFL methods by up to 3.52\%.

Related papers

• Beyond Point-Wise Matching: Structural Representation Alignment for Accelerating Diffusion Transformers [93.3976834364707]
  We propose sREPA, a structural REPresentation Alignment framework to enforce consistency in the relational geometry of feature maps.<n>By encouraging the model to internalize holistic spatial layouts and structural correlations from pre-trained features, sREPA achieves faster and more stable convergence.
  arXiv  Detail & Related papers  (2026-05-16T12:01:04Z)
• FedHPro: Federated Hyper-Prototype Learning via Gradient Matching [34.44446063708919]
  Federated Learning (FL) enables collaborative training of distributed clients while protecting privacy.<n>We introduce hyper-prototypes, defined by a set of learnable global class-wise prototypes to preserve semantic knowledge across clients.<n>We propose FedHPro, a Federated Hyper-Prototype Learning framework, to leverage hyper-prototypes to promote inter-class separability.
  arXiv  Detail & Related papers  (2026-05-13T13:01:17Z)
• Beyond Rigid Alignment: Graph Federated Learning via Dual Manifold Calibration [35.251447232161404]
  Graph Federated Learning (GFL) enables collaborative representation learning across distributed subgraphs while preserving privacy.<n>We propose Federated Graph Manifold (FedGMC), a novel paradigm that tackles semantic heterogeneity from a unified manifold perspective.<n>We show that FedGMC effectively balances global commonality and local personalization, thereby significantly outperforming state-of-the-art baseline methods.
  arXiv  Detail & Related papers  (2026-05-07T13:35:55Z)
• Replacing Parameters with Preferences: Federated Alignment of Heterogeneous Vision-Language Models [63.70401095689976]
  Vision-Language Models (VLMs) have broad potential in privacy-sensitive domains such as healthcare and finance.<n>We propose MoR, a federated alignment framework that combines GRPO with Mixture-of-Rewards for heterogeneousVLMs.<n>MoR consistently outperforms federated alignment baselines in generalization and cross-client adaptability.
  arXiv  Detail & Related papers  (2026-05-05T07:02:50Z)
• Empowering Heterogeneous Graph Foundation Models via Decoupled Relation Alignment [20.128308877213044]
  Decoupled relation Subspace Alignment (DRSA) is a novel, plug-and-play relation-driven framework.<n>It fundamentally shifts the paradigm by decoupling feature semantics from relation structures.<n>It can be seamlessly integrated as a universal preprocessing module.
  arXiv  Detail & Related papers  (2026-05-01T15:31:36Z)
• H2Tune: Federated Foundation Model Fine-Tuning with Hybrid Heterogeneity [25.639893635287187]
  We propose H2Tune, a federated foundation model fine-tuning with hybrid heterogeneity.<n>Our framework consists of three key components: (i) sparsified triple matrix decomposition to align hidden dimensions across clients through constructing rank-consistent middle matrices, with adaptive sparsification based on client resources; (ii) relation-guided matrix layer alignment to handle heterogeneous layer structures and representation capabilities; and (iii) alternating task-knowledge disentanglement mechanism to decouple shared and specific knowledge of local model parameters through alternating optimization.
  arXiv  Detail & Related papers  (2025-07-30T12:53:18Z)
• Cross-architecture universal feature coding via distribution alignment [88.73189953617594]
  We introduce a new research problem: cross-architecture universal feature coding (CAUFC)<n>We propose a two-step distribution alignment method. First, we design the format alignment method that CNN and Transformer features into a consistent 2D token format. Second, we propose the feature value alignment method that harmonizes statistical distributions via truncation and normalization.<n>As a first attempt to study CAUFC, we evaluate our method on the image classification task. Experimental results demonstrate that our method achieves superior rate-accuracy trade-offs compared to the architecture-specific baseline.
  arXiv  Detail & Related papers  (2025-06-15T06:14:02Z)
• A Randomized Zeroth-Order Hierarchical Framework for Heterogeneous Federated Learning [3.797867929356259]
  We propose a novel framework by formulating heterogeneous FL as a hierarchical optimization problem.<n>This new framework captures both local and global training processes through a bilevel formulation.<n>We implement our method on image classification tasks and compare with other methods under different heterogeneous settings.
  arXiv  Detail & Related papers  (2025-04-02T15:44:59Z)
• Rethinking Client Drift in Federated Learning: A Logit Perspective [125.35844582366441]
  Federated Learning (FL) enables multiple clients to collaboratively learn in a distributed way, allowing for privacy protection. We find that the difference in logits between the local and global models increases as the model is continuously updated. We propose a new algorithm, named FedCSD, a Class prototype Similarity Distillation in a federated framework to align the local and global models.
  arXiv  Detail & Related papers  (2023-08-20T04:41:01Z)
• Towards Instance-adaptive Inference for Federated Learning [80.38701896056828]
  Federated learning (FL) is a distributed learning paradigm that enables multiple clients to learn a powerful global model by aggregating local training. In this paper, we present a novel FL algorithm, i.e., FedIns, to handle intra-client data heterogeneity by enabling instance-adaptive inference in the FL framework. Our experiments show that our FedIns outperforms state-of-the-art FL algorithms, e.g., a 6.64% improvement against the top-performing method with less than 15% communication cost on Tiny-ImageNet.
  arXiv  Detail & Related papers  (2023-08-11T09:58:47Z)
• Disentangled Federated Learning for Tackling Attributes Skew via Invariant Aggregation and Diversity Transferring [104.19414150171472]
  Attributes skews the current federated learning (FL) frameworks from consistent optimization directions among the clients. We propose disentangled federated learning (DFL) to disentangle the domain-specific and cross-invariant attributes into two complementary branches. Experiments verify that DFL facilitates FL with higher performance, better interpretability, and faster convergence rate, compared with SOTA FL methods.
  arXiv  Detail & Related papers  (2022-06-14T13:12:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.