DriftMoE: A Mixture of Experts Approach to Handle Concept Drifts

• URL: http://arxiv.org/abs/2507.18464v1
• Date: Thu, 24 Jul 2025 14:39:20 GMT
• Title: DriftMoE: A Mixture of Experts Approach to Handle Concept Drifts
• Authors: Miguel Aspis, Sebastián A. Cajas Ordónez, Andrés L. Suárez-Cetrulo, Ricardo Simón Carbajo,
• Abstract summary: This paper introduces DriftMoE, an online Mixture-of-Experts (MoE) architecture that addresses limitations through a novel co-training framework.<n> DriftMoE features a compact neural router that is co-trained alongside a pool of incremental Hoeffding tree experts.<n>We evaluate DriftMoE's performance across nine state-of-the-art data stream learning benchmarks spanning abrupt, gradual, and real-world drifts.
• Score: 1.2487037582320804
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Learning from non-stationary data streams subject to concept drift requires models that can adapt on-the-fly while remaining resource-efficient. Existing adaptive ensemble methods often rely on coarse-grained adaptation mechanisms or simple voting schemes that fail to optimally leverage specialized knowledge. This paper introduces DriftMoE, an online Mixture-of-Experts (MoE) architecture that addresses these limitations through a novel co-training framework. DriftMoE features a compact neural router that is co-trained alongside a pool of incremental Hoeffding tree experts. The key innovation lies in a symbiotic learning loop that enables expert specialization: the router selects the most suitable expert for prediction, the relevant experts update incrementally with the true label, and the router refines its parameters using a multi-hot correctness mask that reinforces every accurate expert. This feedback loop provides the router with a clear training signal while accelerating expert specialization. We evaluate DriftMoE's performance across nine state-of-the-art data stream learning benchmarks spanning abrupt, gradual, and real-world drifts testing two distinct configurations: one where experts specialize on data regimes (multi-class variant), and another where they focus on single-class specialization (task-based variant). Our results demonstrate that DriftMoE achieves competitive results with state-of-the-art stream learning adaptive ensembles, offering a principled and efficient approach to concept drift adaptation. All code, data pipelines, and reproducibility scripts are available in our public GitHub repository: https://github.com/miguel-ceadar/drift-moe.

Related papers

• FedDAA: Dynamic Client Clustering for Concept Drift Adaptation in Federated Learning [6.327777127276663]
  We propose FedDAA, a dynamic clustered FL framework designed to adapt to multi-source concept drift while preserving valuable historical knowledge.<n>Specifically, FedDAA integrates three modules: a cluster number determination module to find the optimal number of clusters; a real drift detection module to distinguish real drift from virtual/label drift; and a concept drift adaptation module to adapt to new data while retaining useful historical information.<n>Experiments show that FedDAA achieves 7.84% to 8.52% accuracy improvements over state-of-the-art methods on Fashion-MNIST, CIFAR-10, and CIFAR-100.
  arXiv  Detail & Related papers  (2025-06-26T07:09:08Z)
• XTrack: Multimodal Training Boosts RGB-X Video Object Trackers [88.72203975896558]
  It is crucial to ensure that knowledge gained from multimodal sensing is effectively shared.<n>Similar samples across different modalities have more knowledge to share than otherwise.<n>We propose a method for RGB-X tracker during inference, with an average +3% precision improvement over the current SOTA.
  arXiv  Detail & Related papers  (2024-05-28T03:00:58Z)
• Methods for Generating Drift in Text Streams [49.3179290313959]
  Concept drift is a frequent phenomenon in real-world datasets and corresponds to changes in data distribution over time. This paper provides four textual drift generation methods to ease the production of datasets with labeled drifts. Results show that all methods have their performance degraded right after the drifts, and the incremental SVM is the fastest to run and recover the previous performance levels.
  arXiv  Detail & Related papers  (2024-03-18T23:48:33Z)
• A comprehensive analysis of concept drift locality in data streams [3.5897534810405403]
  Concept drift must be detected for effective model adaptation to evolving data properties. We present a novel categorization of concept drift based on its locality and scale. We conduct a comparative assessment of 9 state-of-the-art drift detectors across diverse difficulties.
  arXiv  Detail & Related papers  (2023-11-10T20:57:43Z)
• FEAMOE: Fair, Explainable and Adaptive Mixture of Experts [9.665417053344614]
  We propose FEAMOE, a "mixture-of-experts" inspired framework aimed at learning fairer, more explainable/interpretable models. We show that our framework as applied to a mixture of linear experts is able to perform comparably to neural networks in terms of accuracy while producing fairer models. We also prove that the proposed framework allows for producing fast Shapley value explanations.
  arXiv  Detail & Related papers  (2022-10-10T20:02:02Z)
• Physics-Inspired Temporal Learning of Quadrotor Dynamics for Accurate Model Predictive Trajectory Tracking [76.27433308688592]
  Accurately modeling quadrotor's system dynamics is critical for guaranteeing agile, safe, and stable navigation. We present a novel Physics-Inspired Temporal Convolutional Network (PI-TCN) approach to learning quadrotor's system dynamics purely from robot experience. Our approach combines the expressive power of sparse temporal convolutions and dense feed-forward connections to make accurate system predictions.
  arXiv  Detail & Related papers  (2022-06-07T13:51:35Z)
• Asynchronous Federated Learning for Sensor Data with Concept Drift [17.390098048134195]
  Federated learning (FL) involves multiple distributed devices jointly training a shared model. Most of previous FL approaches assume that data on devices are fixed and stationary during the training process. concept drift makes the learning process complicated because of the inconsistency between existing and upcoming data. We propose a novel approach, FedConD, to detect and deal with the concept drift on local devices.
  arXiv  Detail & Related papers  (2021-09-01T02:06:42Z)
• A Generalizable Model-and-Data Driven Approach for Open-Set RFF Authentication [74.63333951647581]
  Radio-frequency fingerprints(RFFs) are promising solutions for realizing low-cost physical layer authentication. Machine learning-based methods have been proposed for RFF extraction and discrimination. We propose a new end-to-end deep learning framework for extracting RFFs from raw received signals.
  arXiv  Detail & Related papers  (2021-08-10T03:59:37Z)
• Concept drift detection and adaptation for federated and continual learning [55.41644538483948]
  Smart devices can collect vast amounts of data from their environment. This data is suitable for training machine learning models, which can significantly improve their behavior. In this work, we present a new method, called Concept-Drift-Aware Federated Averaging.
  arXiv  Detail & Related papers  (2021-05-27T17:01:58Z)
• Learning Parameter Distributions to Detect Concept Drift in Data Streams [13.20231558027132]
  We propose a novel framework for the detection of real concept drift, called ERICS. By treating the parameters of a predictive model as random variables, we show that concept drift corresponds to a change in the distribution of optimal parameters. ERICS is also capable to detect concept drift at the input level, which is a significant advantage over existing approaches.
  arXiv  Detail & Related papers  (2020-10-19T11:19:16Z)
• Learning From Multiple Experts: Self-paced Knowledge Distillation for Long-tailed Classification [106.08067870620218]
  We propose a self-paced knowledge distillation framework, termed Learning From Multiple Experts (LFME) We refer to these models as 'Experts', and the proposed LFME framework aggregates the knowledge from multiple 'Experts' to learn a unified student model. We conduct extensive experiments and demonstrate that our method is able to achieve superior performances compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-01-06T12:57:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.