NEST: Tackling Dataset-Level Distribution Shifts via Regime-Oriented Mixture-of-Experts

• URL: http://arxiv.org/abs/2607.06607v1
• Date: Tue, 07 Jul 2026 05:36:47 GMT
• Title: NEST: Tackling Dataset-Level Distribution Shifts via Regime-Oriented Mixture-of-Experts
• Abstract summary: NEST is a specialized framework designed to model and recompose datasets as composites of distinct operational regimes.<n>NEST first facilitates structural specialization by partitioning the dataset into distinct operational regimes.<n>Individual experts function as specialized kernels that capture regime-specific dynamics.
• Score: 25.126587044094716
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Accurate long-term forecasting in complex systems is frequently compromised by dataset-level distribution shifts, where diverse underlying behavioral modes and evolving system states drive the dynamic multivariate time-series. While existing methods predominantly focus on local temporal shifts, they fail to explicitly model the global structural challenge where datasets are composites of distinct operational regimes. In this paper, we propose NEST, a specialized framework designed to model and recompose these evolving structures through a two-phase dense MoE architecture. NEST first facilitates structural specialization by partitioning the dataset into distinct operational regimes through unsupervised clustering in a principled moment-entropy space. We introduce a regime-oriented router mechanism that generates initial expert weights based on temporal content, subsequently refined through geometric modulation to regime centroids. Crucially, rather than acting as monolithic predictors, individual experts function as specialized kernels that capture regime-specific dynamics by evolving unique variate-attention patterns. Extensive evaluations on diverse benchmarks, including heterogeneous network traffic and physical phenomena, demonstrate that NEST consistently achieves state-of-the-art performance. Our code and datasets are available at https://github.com/Aaralshin/NEST

Related papers

• Learning Spatio-Temporal Foundation Models from Pure Synthetic Data [85.23980441738811]
  NeoST is a scalable synthetic-training corpus to real-world bias, a latent-space reasoning architecture that generates and iteratively refines multiple future trajectories, and latent-space objectives that emphasize structural dynamics and enable inference-time correction under distribution shifts.<n>Extensive experiments across diverse real-world benchmarks show that NeoST consistently outperforms existing STFMs in diverse real-worldtemporal systems.
  arXiv  Detail & Related papers  (2026-06-27T02:56:46Z)
• UPLOTS: A Unified Pretrained Language Model for Constrained Time-series Generation [9.651308133504818]
  We propose UPLOTS, a Unified, Prompt-guided Language framework fOr constrained Time-Series Generation.<n>Instead of building task-specific models, UPLOTS leverages a single pre-trained transformer backbone guided by learned constraint prompts.<n>We evaluate UPLOTS on four real-world benchmarks and multiple constraint settings, including peak-period, calendar, load-level, and volatility patterns.
  arXiv  Detail & Related papers  (2026-06-09T06:36:06Z)
• SPaRSe-TIME: Saliency-Projected Low-Rank Temporal Modeling for Efficient and Interpretable Time Series Prediction [0.0]
  SPaRSe-TIME is a framework that models time series through a decomposition into three complementary components: saliency, memory, and trend.<n>Experiments on diverse real-world datasets demonstrate that SPaRSe-TIME achieves competitive performance.<n>Overall, SPaRSe-TIME offers a principled alternative to monolithic sequence models, bridging efficiency, interpretability, and performance, and providing a scalable framework for time series learning.
  arXiv  Detail & Related papers  (2026-04-19T09:55:28Z)
• On the Identifiability of Regime-Switching Models with Multi-Lag Dependencies [42.274922469778545]
  We develop a general theoretical framework for multi-lag Regime-Switching Models (RSMs)<n>We prove identifiability of both the number of regimes and the multi-lag transitions in a nonlinear-Gaussian setting.<n>Our results hold in a fully unsupervised setting through architectural and noise assumptions.
  arXiv  Detail & Related papers  (2026-01-06T15:12:39Z)
• UniDiff: A Unified Diffusion Framework for Multimodal Time Series Forecasting [90.47915032778366]
  We propose UniDiff, a unified diffusion framework for multimodal time series forecasting.<n>At its core lies a unified and parallel fusion module, where a single cross-attention mechanism integrates structural information from timestamps and semantic context from texts.<n>Experiments on real-world benchmark datasets across eight domains demonstrate that the proposed UniDiff model achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-12-08T05:36:14Z)
• Adaptive Fine-Tuning via Pattern Specialization for Deep Time Series Forecasting [0.8901073744693314]
  Time series forecasting poses significant challenges in non-stationary environments.<n>We propose a novel framework that enhances deep neural network (DNN) performance by leveraging specialized model adaptation and selection.
  arXiv  Detail & Related papers  (2025-08-11T12:40:08Z)
• STAR-Rec: Making Peace with Length Variance and Pattern Diversity in Sequential Recommendation [61.320991769685065]
  STAR-Rec is a novel architecture that combines preference-aware attention and state-space modeling.<n>We show that STAR-Rec consistently outperforms state-of-the-art sequential recommendation methods.
  arXiv  Detail & Related papers  (2025-05-06T12:40:38Z)
• Learning Interpretable Hierarchical Dynamical Systems Models from Time Series Data [6.3128614613706295]
  We introduce a hierarchical framework that enables to harvest group-level (multi-domain) information while retaining single-domain characteristics.<n>In addition to faithful reconstruction of all individual dynamical regimes, our unsupervised methodology discovers common low-dimensional feature spaces.
  arXiv  Detail & Related papers  (2024-10-07T07:54:53Z)
• Attractor Memory for Long-Term Time Series Forecasting: A Chaos Perspective [63.60312929416228]
  textbftextitAttraos incorporates chaos theory into long-term time series forecasting. We show that Attraos outperforms various LTSF methods on mainstream datasets and chaotic datasets with only one-twelfth of the parameters compared to PatchTST.
  arXiv  Detail & Related papers  (2024-02-18T05:35:01Z)
• Causal Temporal Regime Structure Learning [49.77103348208835]
  We present CASTOR, a novel method that concurrently learns the Directed Acyclic Graph (DAG) for each regime.<n>We establish the identifiability of the regimes and DAGs within our framework.<n>Experiments show that CASTOR consistently outperforms existing causal discovery models.
  arXiv  Detail & Related papers  (2023-11-02T17:26:49Z)
• Disentangling Structured Components: Towards Adaptive, Interpretable and Scalable Time Series Forecasting [52.47493322446537]
  We develop a adaptive, interpretable and scalable forecasting framework, which seeks to individually model each component of the spatial-temporal patterns. SCNN works with a pre-defined generative process of MTS, which arithmetically characterizes the latent structure of the spatial-temporal patterns. Extensive experiments are conducted to demonstrate that SCNN can achieve superior performance over state-of-the-art models on three real-world datasets.
  arXiv  Detail & Related papers  (2023-05-22T13:39:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.