Mixture of Predefined Experts: Maximizing Data Usage on Vertical Federated Learning

• URL: http://arxiv.org/abs/2602.12708v1
• Date: Fri, 13 Feb 2026 08:21:20 GMT
• Title: Mixture of Predefined Experts: Maximizing Data Usage on Vertical Federated Learning
• Authors: Jon Irureta, Gorka Azkune, Jon Imaz, Aizea Lojo, Javier Fernandez-Marques,
• Abstract summary: Split-MoPE is a novel framework that integrates Split Learning with a specialized Mixture of Predefined Experts architecture.<n>MoPE uses predefined experts to process specific data alignments, effectively maximizing data usage during both training and inference.<n>By leveraging pretrained encoders for target data domains, Split-MoPE achieves state-of-the-art performance in a single communication round.
• Score: 4.1346825639910785
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vertical Federated Learning (VFL) has emerged as a critical paradigm for collaborative model training in privacy-sensitive domains such as finance and healthcare. However, most existing VFL frameworks rely on the idealized assumption of full sample alignment across participants, a premise that rarely holds in real-world scenarios. To bridge this gap, this work introduces Split-MoPE, a novel framework that integrates Split Learning with a specialized Mixture of Predefined Experts (MoPE) architecture. Unlike standard Mixture of Experts (MoE), where routing is learned dynamically, MoPE uses predefined experts to process specific data alignments, effectively maximizing data usage during both training and inference without requiring full sample overlap. By leveraging pretrained encoders for target data domains, Split-MoPE achieves state-of-the-art performance in a single communication round, significantly reducing the communication footprint compared to multi-round end-to-end training. Furthermore, unlike existing proposals that address sample misalignment, this novel architecture provides inherent robustness against malicious or noisy participants and offers per-sample interpretability by quantifying each collaborator's contribution to each prediction. Extensive evaluations on vision (CIFAR-10/100) and tabular (Breast Cancer Wisconsin) datasets demonstrate that Split-MoPE consistently outperforms state-of-the-art systems such as LASER and Vertical SplitNN, particularly in challenging scenarios with high data missingness.

Related papers

• Benchmarking Few-shot Transferability of Pre-trained Models with Improved Evaluation Protocols [123.73663884421272]
  Few-shot transfer has been revolutionized by stronger pre-trained models and improved adaptation algorithms.<n>We establish FEWTRANS, a comprehensive benchmark containing 10 diverse datasets.<n>By releasing FEWTRANS, we aim to provide a rigorous "ruler" to streamline reproducible advances in few-shot transfer learning research.
  arXiv  Detail & Related papers  (2026-02-28T05:41:57Z)
• Personalized Subgraph Federated Learning with Differentiable Auxiliary Projections [15.488985833084408]
  We introduce Federated learning with Auxiliary projections (FedAux)<n>FedAux is a personalized subgraph FL framework that learns to align, compare, and aggregate heterogeneously distributed local models without sharing raw data or node embeddings.<n> Empirical evaluations across diverse graph benchmarks demonstrate that FedAux substantially outperforms existing baselines in both accuracy and personalization performance.
  arXiv  Detail & Related papers  (2025-05-29T09:17:49Z)
• A Novel Algorithm for Personalized Federated Learning: Knowledge Distillation with Weighted Combination Loss [0.0]
  pFedKD-WCL integrates knowledge distillation with bi-level optimization to address non-IID challenges.<n>We evaluate pFedKD-WCL on the MNIST dataset and a synthetic dataset with non-IID, using multinomial logistic regression and multilayer perceptron models.
  arXiv  Detail & Related papers  (2025-04-06T23:22:03Z)
• Hybrid-Regularized Magnitude Pruning for Robust Federated Learning under Covariate Shift [2.298932494750101]
  We show that inconsistencies in client-side training distributions substantially degrade the performance of federated learning models.<n>We propose a novel FL framework using a combination of pruning and regularisation of clients' training to improve the sparsity, redundancy, and robustness of neural connections.
  arXiv  Detail & Related papers  (2024-12-19T16:22:37Z)
• Leveraging Foundation Models for Multi-modal Federated Learning with Incomplete Modality [41.79433449873368]
  We propose a novel multi-modal federated learning method, Federated Multi-modal contrastiVe training with Pre-trained completion (FedMVP) FedMVP integrates the large-scale pre-trained models to enhance the federated training. We demonstrate that the model achieves superior performance over two real-world image-text classification datasets.
  arXiv  Detail & Related papers  (2024-06-16T19:18:06Z)
• Federated Learning with Projected Trajectory Regularization [65.6266768678291]
  Federated learning enables joint training of machine learning models from distributed clients without sharing their local data. One key challenge in federated learning is to handle non-identically distributed data across the clients. We propose a novel federated learning framework with projected trajectory regularization (FedPTR) for tackling the data issue.
  arXiv  Detail & Related papers  (2023-12-22T02:12:08Z)
• Personalized Federated Learning under Mixture of Distributions [98.25444470990107]
  We propose a novel approach to Personalized Federated Learning (PFL), which utilizes Gaussian mixture models (GMM) to fit the input data distributions across diverse clients. FedGMM possesses an additional advantage of adapting to new clients with minimal overhead, and it also enables uncertainty quantification. Empirical evaluations on synthetic and benchmark datasets demonstrate the superior performance of our method in both PFL classification and novel sample detection.
  arXiv  Detail & Related papers  (2023-05-01T20:04:46Z)
• Personalizing Federated Learning with Over-the-Air Computations [84.8089761800994]
  Federated edge learning is a promising technology to deploy intelligence at the edge of wireless networks in a privacy-preserving manner. Under such a setting, multiple clients collaboratively train a global generic model under the coordination of an edge server. This paper presents a distributed training paradigm that employs analog over-the-air computation to address the communication bottleneck.
  arXiv  Detail & Related papers  (2023-02-24T08:41:19Z)
• DRFLM: Distributionally Robust Federated Learning with Inter-client Noise via Local Mixup [58.894901088797376]
  federated learning has emerged as a promising approach for training a global model using data from multiple organizations without leaking their raw data. We propose a general framework to solve the above two challenges simultaneously. We provide comprehensive theoretical analysis including robustness analysis, convergence analysis, and generalization ability.
  arXiv  Detail & Related papers  (2022-04-16T08:08:29Z)
• Efficient Split-Mix Federated Learning for On-Demand and In-Situ Customization [107.72786199113183]
  Federated learning (FL) provides a distributed learning framework for multiple participants to collaborate learning without sharing raw data. In this paper, we propose a novel Split-Mix FL strategy for heterogeneous participants that, once training is done, provides in-situ customization of model sizes and robustness.
  arXiv  Detail & Related papers  (2022-03-18T04:58:34Z)
• DiVA: Diverse Visual Feature Aggregation for Deep Metric Learning [83.48587570246231]
  Visual Similarity plays an important role in many computer vision applications. Deep metric learning (DML) is a powerful framework for learning such similarities. We propose and study multiple complementary learning tasks, targeting conceptually different data relationships. We learn a single model to aggregate their training signals, resulting in strong generalization and state-of-the-art performance.
  arXiv  Detail & Related papers  (2020-04-28T12:26:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.