Federated Multi-Objective Learning with Controlled Pareto Frontiers

• URL: http://arxiv.org/abs/2508.05424v3
• Date: Sat, 23 Aug 2025 01:32:27 GMT
• Title: Federated Multi-Objective Learning with Controlled Pareto Frontiers
• Authors: Jiansheng Rao, Jiayi Li, Zhizhi Gong, Soummya Kar, Haoxuan Li,
• Abstract summary: Federated learning (FL) is a widely adopted paradigm for privacy-preserving model training.<n>Existing methods such as federated multi-objective learning (FMOL) attempts to import multi-objective optimisation (MOO) into FL.<n>We introduce Conically-Regularised FMOL (CR-FMOL), the first federated MOO framework that enforces client-wise optimality through a preference-cone constraint.
• Score: 10.818539304970935
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Federated learning (FL) is a widely adopted paradigm for privacy-preserving model training, but FedAvg optimise for the majority while under-serving minority clients. Existing methods such as federated multi-objective learning (FMOL) attempts to import multi-objective optimisation (MOO) into FL. However, it merely delivers task-wise Pareto-stationary points, leaving client fairness to chance. In this paper, we introduce Conically-Regularised FMOL (CR-FMOL), the first federated MOO framework that enforces client-wise Pareto optimality through a novel preference-cone constraint. After local federated multi-gradient descent averaging (FMGDA) / federated stochastic multi-gradient descent averaging (FSMGDA) steps, each client transmits its aggregated task-loss vector as an implicit preference; the server then solves a cone-constrained Pareto-MTL sub-problem centred at the uniform vector, producing a descent direction that is Pareto-stationary for every client within its cone. Experiments on non-IID benchmarks show that CR-FMOL enhances client fairness, and although the early-stage performance is slightly inferior to FedAvg, it is expected to achieve comparable accuracy given sufficient training rounds.

Related papers

• FIRM: Federated In-client Regularized Multi-objective Alignment for Large Language Models [19.266791462067655]
  We introduce FIRM (Federated In-client Regularized Multi-objective alignment), a novel algorithm that achieves both client disagreement drift mitigation and communication efficiency.<n>We show that FIRM leads to smoother training dynamics, reduced client disagreement drift, and improved reward trade-offs compared to baselines.
  arXiv  Detail & Related papers  (2025-11-21T06:59:28Z)
• Towards Optimal Heterogeneous Client Sampling in Multi-Model Federated Learning [22.787635207005884]
  Federated learning allows edge devices to collaboratively train models without sharing local data.<n>Clients may need to train multiple unrelated FL models, but communication constraints limit their ability to train all models simultaneously.<n>We propose MMFL-LVR, a loss-based sampling method that minimizes training variance while explicitly respecting communication limits at the server.
  arXiv  Detail & Related papers  (2025-04-07T14:43:17Z)
• Client-Centric Federated Adaptive Optimization [78.30827455292827]
  Federated Learning (FL) is a distributed learning paradigm where clients collaboratively train a model while keeping their own data private.<n>We propose Federated-Centric Adaptive Optimization, which is a class of novel federated optimization approaches.
  arXiv  Detail & Related papers  (2025-01-17T04:00:50Z)
• Over-the-Air Fair Federated Learning via Multi-Objective Optimization [52.295563400314094]
  We propose an over-the-air fair federated learning algorithm (OTA-FFL) to train fair FL models.<n>Experiments demonstrate the superiority of OTA-FFL in achieving fairness and robust performance.
  arXiv  Detail & Related papers  (2025-01-06T21:16:51Z)
• Debiasing Federated Learning with Correlated Client Participation [25.521881752822164]
  This paper introduces a theoretical framework that models client participation in FL as a Markov chain. Every client must wait a minimum number of $R$ rounds (minimum separation) before re-participating. We develop an effective debiasing algorithm for FedAvg that provably converges to the unbiased optimal solution.
  arXiv  Detail & Related papers  (2024-10-02T03:30:53Z)
• Personalized Federated Learning under Mixture of Distributions [98.25444470990107]
  We propose a novel approach to Personalized Federated Learning (PFL), which utilizes Gaussian mixture models (GMM) to fit the input data distributions across diverse clients. FedGMM possesses an additional advantage of adapting to new clients with minimal overhead, and it also enables uncertainty quantification. Empirical evaluations on synthetic and benchmark datasets demonstrate the superior performance of our method in both PFL classification and novel sample detection.
  arXiv  Detail & Related papers  (2023-05-01T20:04:46Z)
• Beyond ADMM: A Unified Client-variance-reduced Adaptive Federated Learning Framework [82.36466358313025]
  We propose a primal-dual FL algorithm, termed FedVRA, that allows one to adaptively control the variance-reduction level and biasness of the global model. Experiments based on (semi-supervised) image classification tasks demonstrate superiority of FedVRA over the existing schemes.
  arXiv  Detail & Related papers  (2022-12-03T03:27:51Z)
• Fed-CBS: A Heterogeneity-Aware Client Sampling Mechanism for Federated Learning via Class-Imbalance Reduction [76.26710990597498]
  We show that the class-imbalance of the grouped data from randomly selected clients can lead to significant performance degradation. Based on our key observation, we design an efficient client sampling mechanism, i.e., Federated Class-balanced Sampling (Fed-CBS) In particular, we propose a measure of class-imbalance and then employ homomorphic encryption to derive this measure in a privacy-preserving way.
  arXiv  Detail & Related papers  (2022-09-30T05:42:56Z)
• Low-Latency Federated Learning over Wireless Channels with Differential Privacy [142.5983499872664]
  In federated learning (FL), model training is distributed over clients and local models are aggregated by a central server. In this paper, we aim to minimize FL training delay over wireless channels, constrained by overall training performance as well as each client's differential privacy (DP) requirement.
  arXiv  Detail & Related papers  (2021-06-20T13:51:18Z)
• Distributed Non-Convex Optimization with Sublinear Speedup under Intermittent Client Availability [46.85205907718874]
  Federated learning is a new machine learning framework, where a bunch of clients collaboratively train a model without sharing training data. In this work, we consider a practical and issue when deploying federated learning in intermittent mobile environments. We propose a simple distributed nonlinear optimization algorithm, called Federated Latest Averaging (FedLaAvg for short) Our theoretical analysis shows that FedLaAvg attains the convergence rate of $(E1/2/(NT1/2)$, achieving a sublinear speed with respect to the total number of clients.
  arXiv  Detail & Related papers  (2020-02-18T06:32:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.