Manifold-Aware Temporal Domain Generalization for Large Language Models

• URL: http://arxiv.org/abs/2602.11965v1
• Date: Thu, 12 Feb 2026 14:00:44 GMT
• Title: Manifold-Aware Temporal Domain Generalization for Large Language Models
• Abstract summary: temporal distribution shifts are pervasive in real-world deployments of Large Language Models (LLMs)<n>This paper introduces a geometric reformulation of TDG under parameter-efficient fine-tuning.<n>We propose Manifold-aware Temporal LoRA (MaT-LoRA), which constrains temporal updates to a shared low-dimensional manifold within a low-rank adaptation subspace.
• Score: 17.44312895876345
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Temporal distribution shifts are pervasive in real-world deployments of Large Language Models (LLMs), where data evolves continuously over time. While Temporal Domain Generalization (TDG) seeks to model such structured evolution, existing approaches characterize model adaptation in the full parameter space. This formulation becomes computationally infeasible for modern LLMs. This paper introduces a geometric reformulation of TDG under parameter-efficient fine-tuning. We establish that the low-dimensional temporal structure underlying model evolution can be preserved under parameter-efficient reparameterization, enabling temporal modeling without operating in the ambient parameter space. Building on this principle, we propose Manifold-aware Temporal LoRA (MaT-LoRA), which constrains temporal updates to a shared low-dimensional manifold within a low-rank adaptation subspace, and models its evolution through a structured temporal core. This reparameterization dramatically reduces temporal modeling complexity while retaining expressive power. Extensive experiments on synthetic and real-world datasets, including scientific documents, news publishers, and review ratings, demonstrate that MaT-LoRA achieves superior temporal generalization performance with practical scalability for LLMs.

Related papers

• Learning Spatio-Temporal Foundation Models from Pure Synthetic Data [85.23980441738811]
  NeoST is a scalable synthetic-training corpus to real-world bias, a latent-space reasoning architecture that generates and iteratively refines multiple future trajectories, and latent-space objectives that emphasize structural dynamics and enable inference-time correction under distribution shifts.<n>Extensive experiments across diverse real-world benchmarks show that NeoST consistently outperforms existing STFMs in diverse real-worldtemporal systems.
  arXiv  Detail & Related papers  (2026-06-27T02:56:46Z)
• FRACTAL: SSM with Fractional Recurrent Architecture for Computational Temporal Analysis of Long Sequences [13.320173396455226]
  This paper proposes a Fractional Recurrent Architecture for Computational Temporal Analysis of Long sequences (FRACTAL)<n>By deriving projection operators with analytically characterized spectral properties and a tunable index, the proposed method sensitivity to recent signal perturbations while preserving the spectral structure that encodes fractional scale-invariant memory dynamics.<n>FRACTAL achieves an average score of 87.11% on the Long Range Arena benchmark, including 61.85% on the ListOps task, outperforming the S5 model.
  arXiv  Detail & Related papers  (2026-05-09T09:38:44Z)
• PIMSM: Physics-Informed Multi-Scale Mamba for Stable Neural Representations under Distribution Shift [16.230225739605316]
  We argue that failure to preserve scientific temporal structure contributes to brittleness under distribution shift.<n>We propose Physics-Informed Multi-Scale Mamba, a state-space architecture that maps spectrum-estimated transition points between frequency regimes.<n>On Human Connectome Project fMRI, PIMSM improves robustness and representation stability under severe temporal-context truncation.
  arXiv  Detail & Related papers  (2026-05-08T04:58:05Z)
• MEMTS: Internalizing Domain Knowledge via Parameterized Memory for Retrieval-Free Domain Adaptation of Time Series Foundation Models [51.506429027626005]
  Memory for Time Series (MEMTS) is a lightweight and plug-and-play method for retrieval-free domain adaptation in time series forecasting.<n>Key component of MEMTS is a Knowledge Persistence Module (KPM), which internalizes domain-specific temporal dynamics.<n>This paradigm shift enables MEMTS to achieve accurate domain adaptation with constant-time inference and near-zero latency.
  arXiv  Detail & Related papers  (2026-02-14T14:00:06Z)
• Scalable Spatio-Temporal SE(3) Diffusion for Long-Horizon Protein Dynamics [51.85385061275941]
  Molecular dynamics (MD) simulations remain the gold standard for studying protein dynamics.<n>Recent generative models have shown promise in accelerating simulations, yet they struggle with long-horizon generation.<n>We present STAR-MD, a scalable diffusion model that generates physically plausible protein trajectories over micro-scale timescales.
  arXiv  Detail & Related papers  (2026-02-02T14:13:28Z)
• Enhancing few-shot time series forecasting with LLM-guided diffusion [12.286204074670236]
  Time series forecasting in specialized domains is often constrained by limited data availability.<n>We propose LTSM-DIFF, a novel learning framework that integrates the expressive power of large language models with the generative capability of diffusion models.<n>Our work establishes a new paradigm for time series analysis under data scarcity.
  arXiv  Detail & Related papers  (2026-01-19T06:30:05Z)
• High-Rank Structured Modulation for Parameter-Efficient Fine-Tuning [57.85676271833619]
  Low-rank Adaptation (LoRA) uses a low-rank update method to simulate full parameter fine-tuning.<n>We present textbfSMoA, a high-rank textbfStructured textbfMOdulation textbfAdapter that uses fewer trainable parameters while maintaining a higher rank.
  arXiv  Detail & Related papers  (2026-01-12T13:06:17Z)
• Learning Time-Aware Causal Representation for Model Generalization in Evolving Domains [50.66049136093248]
  We develop a time-aware structural causal model (SCM) that incorporates dynamic causal factors and the causal mechanism drifts.<n>We show that our method can yield the optimal causal predictor for each time domain.<n>Results on both synthetic and real-world datasets exhibit that SYNC can achieve superior temporal generalization performance.
  arXiv  Detail & Related papers  (2025-06-21T14:05:37Z)
• LLM4FTS: Enhancing Large Language Models for Financial Time Series Prediction [0.0]
  Traditional machine learning models exhibit limitations in this forecasting task constrained by their restricted model capacity.<n>We propose $LLM4FTS$, a novel framework that enhances temporal sequence modeling through learnable patch segmentation and dynamic wavelet convolution modules.<n>Experiments on real-world financial datasets substantiate the framework's efficacy, demonstrating superior performance in capturing complex market patterns and achieving state-of-the-art results in stock return prediction.
  arXiv  Detail & Related papers  (2025-05-05T06:48:34Z)
• FACTS: A Factored State-Space Framework For World Modelling [24.08175276756845]
  We propose a novel recurrent framework, the textbfFACTored textbfState-space (textbfFACTS) model, for spatial-temporal world modelling.<n>The FACTS framework constructs a graph-memory with a routing mechanism that learns permutable memory representations.<n>It consistently outperforms or matches specialised state-of-the-art models, despite its general-purpose world modelling design.
  arXiv  Detail & Related papers  (2024-10-28T11:04:42Z)
• On latent dynamics learning in nonlinear reduced order modeling [0.6249768559720122]
  We present the novel mathematical framework of latent dynamics models (LDMs) for reduced order modeling of parameterized nonlinear time-dependent PDEs.<n>A time-continuous setting is employed to derive error and stability estimates for the LDM approximation of the full order model (FOM) solution.<n>Deep neural networks approximate the discrete LDM components, while providing a bounded approximation error with respect to the FOM.
  arXiv  Detail & Related papers  (2024-08-27T16:35:06Z)
• Synthetic location trajectory generation using categorical diffusion models [50.809683239937584]
  Diffusion models (DPMs) have rapidly evolved to be one of the predominant generative models for the simulation of synthetic data. We propose using DPMs for the generation of synthetic individual location trajectories (ILTs) which are sequences of variables representing physical locations visited by individuals.
  arXiv  Detail & Related papers  (2024-02-19T15:57:39Z)
• Parsimony or Capability? Decomposition Delivers Both in Long-term Time Series Forecasting [46.63798583414426]
  Long-term time series forecasting (LTSF) represents a critical frontier in time series analysis. Our study demonstrates, through both analytical and empirical evidence, that decomposition is key to containing excessive model inflation. Remarkably, by tailoring decomposition to the intrinsic dynamics of time series data, our proposed model outperforms existing benchmarks.
  arXiv  Detail & Related papers  (2024-01-22T13:15:40Z)
• Generative Modeling with Phase Stochastic Bridges [49.4474628881673]
  Diffusion models (DMs) represent state-of-the-art generative models for continuous inputs. We introduce a novel generative modeling framework grounded in textbfphase space dynamics Our framework demonstrates the capability to generate realistic data points at an early stage of dynamics propagation.
  arXiv  Detail & Related papers  (2023-10-11T18:38:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.