LLMODE: Aligning ODEs with LLMs via Gated Token Injection for Irregular Spatio-Temporal Forecasting

• URL: http://arxiv.org/abs/2608.29640v1
• Date: Sun, 30 Aug 2026 08:02:20 GMT
• Title: LLMODE: Aligning ODEs with LLMs via Gated Token Injection for Irregular Spatio-Temporal Forecasting
• Abstract summary: Large language models (LLMs) have shown promise for irregular-temporal forecasting, but existing approaches often rely on regularly sampled token sequences.<n>We propose LLMODE, a token-efficient framework for representation-temporal forecasting with a frozen backbone.
• Score: 27.848815822218366
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) have shown promise for spatio-temporal forecasting, but existing approaches often rely on regularly sampled token sequences and struggle with irregular observations because of temporal asynchrony, representation-space misalignment, and limited context windows. We propose LLMODE, a token-efficient framework for irregular spatio-temporal forecasting with a frozen LLM backbone. LLMODE first uses a graph-aware ODE encoder to reconstruct irregular graph observations as a continuous-time latent trajectory. A Fixed-Budget Perceiver Resampler then compresses this variable-length trajectory into a fixed number of dynamic memory tokens. In parallel, compact statistical descriptors are encoded and resampled into context memory tokens. A dual-source gated cross-attention module injects both memories into the frozen LLM, enabling controlled utilization of external spatio-temporal evidence. Experiments on three real-world urban datasets and two physical-dynamics benchmarks show competitive overall performance, with clearer advantages under sparse or dynamically complex irregular sampling. Additional evaluations on unseen urban regions further demonstrate strong zero-shot generalization without adaptation.

Related papers

• Beyond Tokenization: Direct Timestep Embedding and Contrastive Alignment for Time-Series Question Answering [13.074536659496362]
  Recent advances in large language models (LLMs) have given rise to time-series question answering (TSQA)<n>TSQA formulates time-series analysis as natural-language question answering.<n>To address this challenge, we propose CADE, a novel framework for TSQA built upon two key components: direct timestep embedding and semantic alignment.
  arXiv  Detail & Related papers  (2026-06-17T12:07:23Z)
• Efficient Long-Context Modeling in Diffusion Language Models via Block Approximate Sparse Attention [58.484607892210015]
  Diffusion Language Models (DLMs) enable globally coherent, bidirectional, and controllable text generation.<n>Many existing block-sparse attention methods select blocks by fixed sampling patterns over the high-resolution attention space.<n>We propose the Block Approximate Sparse Attention framework (BA-Att) with block-wise pre-downsampled operation.
  arXiv  Detail & Related papers  (2026-05-19T12:01:13Z)
• Perceive, Route and Modulate: Dynamic Pattern Recalibration for Time Series Forecasting [68.53528085362642]
  Current deep forecasting models rely on fixed weight matrices applied uniformly to all temporal tokens.<n>We introduce Dynamic Pattern Recalibration, a backbone-agnostic mechanism that resolves this via token-level recalibration.<n>DPRNet achieves competitive performance across 12 benchmarks, validating dynamic recalibration against macroscopic parameter scaling.
  arXiv  Detail & Related papers  (2026-05-07T14:12:47Z)
• LEFT: Learnable Fusion of Tri-view Tokens for Unsupervised Time Series Anomaly Detection [53.191369031661885]
  Unsupervised time series anomaly detection aims to build a model for identifying abnormal timestamps without assuming the availability of annotations.<n>We present Learnable Fusion of Tri-view Tokens (LEFT), a unified unsupervised TSAD framework that models anomalies as inconsistencies across complementary representations.<n>Experiments on real-world benchmarks show that LEFT yields the best detection accuracy against SOTA baselines, while achieving a 5x reduction on FLOPs and 8x speed-up for training.
  arXiv  Detail & Related papers  (2026-02-09T13:33:49Z)
• Self-Rewarding Sequential Monte Carlo for Masked Diffusion Language Models [58.946955321428845]
  This work presents self-rewarding sequential Monte Carlo (SMC)<n>Our algorithm stems from the observation that most existing MDLMs rely on a confidence-based sampling strategy.<n>We introduce the trajectory-level confidence as a self-rewarding signal for assigning particle importance weights.
  arXiv  Detail & Related papers  (2026-02-02T09:21:45Z)
• Enhancing Zero-Shot Time Series Forecasting in Off-the-Shelf LLMs via Noise Injection [18.267727687739853]
  Large Language Models (LLMs) have demonstrated effectiveness as zero-shot time series (TS) forecasters.<n>The key challenge lies in tokenizing TS data into textual representations that align with LLMs' pre-trained knowledge.<n>We introduce two novel TS datasets that fall outside all utilized LLMs' pre-training scopes, and consistently observe improved performance.
  arXiv  Detail & Related papers  (2025-12-23T08:02:33Z)
• SPEAR: Soft Prompt Enhanced Anomaly Recognition for Time Series Data [15.670885465827206]
  Time series anomaly detection plays a crucial role in a wide range of fields, such as healthcare and internet traffic monitoring.<n>The emergence of large language models (LLMs) offers new opportunities for detecting anomalies in the ubiquitous time series data.<n>We propose Soft Prompt Enhanced Anomaly Recognition (SPEAR), a novel approach to leverage LLMs for anomaly detection with soft prompts and quantization.
  arXiv  Detail & Related papers  (2025-10-04T22:20:48Z)
• AXIS: Explainable Time Series Anomaly Detection with Large Language Models [33.68487894996624]
  AXIS is a framework that conditions a frozen Large Language Models (LLMs) for nuanced time-series understanding.<n>LLMs operate on discrete tokens and struggle to directly process long, continuous signals.<n>We introduce a new benchmark featuring multi-format questions and rationales that supervise contextual grounding and pattern-level semantics.
  arXiv  Detail & Related papers  (2025-09-29T07:24:22Z)
• SampleAttention: Near-Lossless Acceleration of Long Context LLM Inference with Adaptive Structured Sparse Attention [53.4441894198495]
  Large language models (LLMs) now support extremely long context windows.<n>The quadratic complexity of vanilla attention results in significantly long Time-to-First-Token (TTFT) latency.<n>We propose SampleAttention, an adaptive structured and near-lossless sparse attention.
  arXiv  Detail & Related papers  (2024-06-17T11:05:15Z)
• Non-autoregressive Sequence-to-Sequence Vision-Language Models [59.445765313094434]
  We propose a parallel decoding sequence-to-sequence vision-language model that marginalizes over multiple inference paths in the decoder.<n>The model achieves performance on-par with its state-of-the-art autoregressive counterpart, but is faster at inference time.
  arXiv  Detail & Related papers  (2024-03-04T17:34:59Z)
• Explaining Time Series via Contrastive and Locally Sparse Perturbations [45.055327583283315]
  ContraLSP is a sparse model that introduces counterfactual samples to build uninformative perturbations but keeps distribution using contrastive learning. Empirical studies on both synthetic and real-world datasets show that ContraLSP outperforms state-of-the-art models.
  arXiv  Detail & Related papers  (2024-01-16T18:27:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.