Replacing Parameters with Preferences: Federated Alignment of Heterogeneous Vision-Language Models

• URL: http://arxiv.org/abs/2602.00485v1
• Date: Sat, 31 Jan 2026 03:11:51 GMT
• Title: Replacing Parameters with Preferences: Federated Alignment of Heterogeneous Vision-Language Models
• Authors: Shule Lu, Yujing Wang, Hainan Zhang, Xiaoshan Yang, Hongwei Zheng, Yongxin Tong, Changsheng Xu, Zhiming Zheng,
• Abstract summary: We argue that replacing parameters with preferences represents a more scalable and privacy-preserving future.<n>We propose MoR, a federated alignment framework based on GRPO with Mixture-of-Rewards for heterogeneous VLMs.<n>MoR consistently outperforms federated alignment baselines in generalization, robustness, and cross-client adaptability.
• Score: 63.70401095689976
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: VLMs have broad potential in privacy-sensitive domains such as healthcare and finance, yet strict data-sharing constraints render centralized training infeasible. FL mitigates this issue by enabling decentralized training, but practical deployments face challenges due to client heterogeneity in computational resources, application requirements, and model architectures. We argue that while replacing data with model parameters characterizes the present of FL, replacing parameters with preferences represents a more scalable and privacy-preserving future. Motivated by this perspective, we propose MoR, a federated alignment framework based on GRPO with Mixture-of-Rewards for heterogeneous VLMs. MoR initializes a visual foundation model as a KL-regularized reference, while each client locally trains a reward model from local preference annotations, capturing specific evaluation signals without exposing raw data. To reconcile heterogeneous rewards, we introduce a routing-based fusion mechanism that adaptively aggregates client reward signals. Finally, the server performs GRPO with this mixed reward to optimize the base VLM. Experiments on three public VQA benchmarks demonstrate that MoR consistently outperforms federated alignment baselines in generalization, robustness, and cross-client adaptability. Our approach provides a scalable solution for privacy-preserving alignment of heterogeneous VLMs under federated settings.

Related papers

• FeDecider: An LLM-Based Framework for Federated Cross-Domain Recommendation [75.50721642765994]
  Large language model (LLM)-based recommendation models have demonstrated impressive performance.<n>We propose an LLM-based framework for Federated cross-domain recommendation, FeDecider.<n>Extensive experiments across diverse datasets validate the effectiveness of our proposed FeDecider.
  arXiv  Detail & Related papers  (2026-02-17T21:42:28Z)
• FedGRPO: Privately Optimizing Foundation Models with Group-Relative Rewards from Domain Client [21.08829811371245]
  Existing methods based on model level or representation level knowledge transfer either require expensive local training or incur high communication costs.<n>We reformulate this problem as a reinforcement learning style evaluation process and propose FedGRPO.<n>FedGRPO achieves superior downstream accuracy and communication efficiency compared to conventional FedFMs baselines.
  arXiv  Detail & Related papers  (2026-02-12T14:45:56Z)
• Adaptive Dual-Weighting Framework for Federated Learning via Out-of-Distribution Detection [53.45696787935487]
  Federated Learning (FL) enables collaborative model training across large-scale distributed service nodes.<n>In real-world service-oriented deployments, data generated by heterogeneous users, devices, and application scenarios are inherently non-IID.<n>We propose FLood, a novel FL framework inspired by out-of-distribution (OOD) detection.
  arXiv  Detail & Related papers  (2026-02-01T05:54:59Z)
• Generalized and Personalized Federated Learning with Foundation Models via Orthogonal Transformations [4.008780119020479]
  Federated Learning aims to train models across decentralized clients or devices holding local data without the need for centralized data collection.<n>We introduce FedOT, a novel approach that leverages black-box foundation models.<n>FedOT mitigates gradient conflicts across diverse clients, preserves semantic integrity, and achieves robust performance even in the presence of substantial data.
  arXiv  Detail & Related papers  (2025-05-26T12:18:24Z)
• Client-Centric Federated Adaptive Optimization [78.30827455292827]
  Federated Learning (FL) is a distributed learning paradigm where clients collaboratively train a model while keeping their own data private.<n>We propose Federated-Centric Adaptive Optimization, which is a class of novel federated optimization approaches.
  arXiv  Detail & Related papers  (2025-01-17T04:00:50Z)
• Hybrid-Regularized Magnitude Pruning for Robust Federated Learning under Covariate Shift [2.298932494750101]
  We show that inconsistencies in client-side training distributions substantially degrade the performance of federated learning models.<n>We propose a novel FL framework using a combination of pruning and regularisation of clients' training to improve the sparsity, redundancy, and robustness of neural connections.
  arXiv  Detail & Related papers  (2024-12-19T16:22:37Z)
• An Architecture Built for Federated Learning: Addressing Data Heterogeneity through Adaptive Normalization-Free Feature Recalibration [0.3481075494213406]
  We propose Adaptive Normalization-free Feature Recalibration (ANFR) to combat heterogeneous data in Federated Learning (FL)<n>ANFR combines weight standardization and channel attention to produce learnable scaling factors for feature maps.<n>Experiments show ANFR consistently outperforms established baselines across various aggregation methods.
  arXiv  Detail & Related papers  (2024-10-02T20:16:56Z)
• Towards Instance-adaptive Inference for Federated Learning [80.38701896056828]
  Federated learning (FL) is a distributed learning paradigm that enables multiple clients to learn a powerful global model by aggregating local training. In this paper, we present a novel FL algorithm, i.e., FedIns, to handle intra-client data heterogeneity by enabling instance-adaptive inference in the FL framework. Our experiments show that our FedIns outperforms state-of-the-art FL algorithms, e.g., a 6.64% improvement against the top-performing method with less than 15% communication cost on Tiny-ImageNet.
  arXiv  Detail & Related papers  (2023-08-11T09:58:47Z)
• Personalized Federated Learning under Mixture of Distributions [98.25444470990107]
  We propose a novel approach to Personalized Federated Learning (PFL), which utilizes Gaussian mixture models (GMM) to fit the input data distributions across diverse clients. FedGMM possesses an additional advantage of adapting to new clients with minimal overhead, and it also enables uncertainty quantification. Empirical evaluations on synthetic and benchmark datasets demonstrate the superior performance of our method in both PFL classification and novel sample detection.
  arXiv  Detail & Related papers  (2023-05-01T20:04:46Z)
• Local-Adaptive Face Recognition via Graph-based Meta-Clustering and Regularized Adaptation [21.08555249703121]
  We introduce a new problem setup called Local-Adaptive Face Recognition (LaFR) LaFR aims at getting optimal performance by training local-adapted models automatically and un-supervisely. We show that LaFR can further improve the global model by a simple federated aggregation over the updated local models.
  arXiv  Detail & Related papers  (2022-03-27T15:20:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.