TIDES: Implicit Time-Awareness in Selective State Space Models

• URL: http://arxiv.org/abs/2605.09742v1
• Date: Sun, 10 May 2026 20:34:06 GMT
• Title: TIDES: Implicit Time-Awareness in Selective State Space Models
• Abstract summary: Continuous-time SSMs handle irregular timestamps, but their dynamics remain linear time-invariant (LTI)<n>We propose textbfTIDES, a selective SSM variant that reconciles selective and continuous architectures.<n> TIDES sets the new state-of-the-art average rank on UEA time-series classification and the Physiome-ODE regression benchmark.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Selective state space models (SSMs), such as Mamba, achieve strong per-token expressivity by making the time discretization step $\TildeΔ$ a learned function of the input. However, in doing so, $\TildeΔ$ ceases to represent a physical sampling interval, limiting its irregular time series modeling capability. Continuous-time SSMs, such as S5, preserve the physical meaning of $\TildeΔ$ and handle irregular timestamps natively ($\TildeΔ\equivΔ)$, but their dynamics remain linear time-invariant (LTI), limiting per-token expressivity. We propose \textbf{TIDES}, a selective SSM variant that reconciles selective and continuous architectures by moving input-dependence off the step size and onto the diagonal state matrix. As a result, $\TildeΔ$ retains its physical meaning, tied to the state discretization, allowing the model to handle irregular timestamps natively without sacrificing the per-token expressivity that makes selective SSMs effective. We show this on a novel \emph{Fading Flash} experimental benchmark, a compact controlled diagnostic for sequence models that jointly tests input-dependence and extrapolation to out-of-distribution $Δ$ values, and isolates the distinct failure modes of current state-of-the-art architectures that TIDES avoids by construction. On large-scale benchmarks, TIDES sets the new state-of-the-art average rank on UEA time-series classification and the Physiome-ODE regression benchmark. Code available at: https://github.com/TaylanSoydan/TIDES.

Related papers

• Deep learning-based prediction of time-resolved adhesive forces in viscoelastic Hertzian contacts [0.0]
  We train a stateful, sequence-to-sequence deep learning model to predict the full force evolution from a prescribed displacement history.<n>The model is tested across unseen parameter combinations and against analytical limiting cases.<n>It provides a rapid surrogate for repeated numerical evaluations with potential use in control-oriented applications.
  arXiv  Detail & Related papers  (2026-07-21T12:48:48Z)
• QuITE: Query-Based Irregular Time Series Embedding [0.0]
  We introduce QuITE (Query-Based Irregular Time Series Embedding), a simple yet effective plug-and-play embedding module for IMTS.<n>QuITE employs learnable query tokens to aggregate irregular observations through a single self-attention layer.<n>Experiments show that QuITE consistently improves MTS models, yielding average relative gains of up to $54.7%$ in forecasting and $15.8%$ in classification.
  arXiv  Detail & Related papers  (2026-05-27T08:48:58Z)
• Do Language Models Need Sleep? Offline Recurrence for Improved Online Inference [58.46323651532913]
  We study a sleep-like consolidation mechanism in which a model periodically converts recent context into persistent fast weights.<n>During sleep, the model performs $N$ offline recurrent passes over the accumulated context and updates the fast weights in its state-space model.<n>We show that increasing sleep duration $N$ for our models improves performance, with the largest gains on examples that require deeper reasoning.
  arXiv  Detail & Related papers  (2026-05-25T17:55:39Z)
• Sessa: Selective State Space Attention [0.0]
  We propose Sessa, a decoder that places attention inside a recurrent feedback path.<n>We prove that Sessa admits power-law memory tails $O(ell-)$ for $0 1$, with slower decay than in the corresponding Transformer and Mamba-style baselines.
  arXiv  Detail & Related papers  (2026-04-20T17:59:08Z)
• CAWN: Continuous Acoustic Wave Networks for Autoregressive Language Modeling [46.16066322190728]
  We introduce the Continuous Acoustic Wave Network (CAWN), a fully continuous sequence-mixing architecture.<n>Instead of discrete matrix-based attention, CAWN projects hidden states into multi-headed complex-domain phasors.<n>To prevent signal degradation over ultra-long contexts, we introduce a dual-gated Selective Phase Resonance mechanism.
  arXiv  Detail & Related papers  (2026-04-05T20:13:22Z)
• Time Is All It Takes: Spike-Retiming Attacks on Event-Driven Spiking Neural Networks [87.16809558673403]
  Spiking neural networks (SNNs) compute with discrete spikes and exploit temporal structure.<n>We study a timing-only adversary that retimes existing spikes while preserving spike counts and amplitudes in event-driven SNNs.
  arXiv  Detail & Related papers  (2026-02-03T09:06:53Z)
• Catch Your Breath: Adaptive Computation for Self-Paced Sequence Production [55.76222360698305]
  We explore a class of supervised training objectives that allow a language model to dynamically and autonomously scale the number of compute steps used for each input token.<n>For any token, the model can request additional compute steps by emitting a don't know> output.<n>We find that the CYB model requests additional steps when doing so improves accuracy, and the model adapts its processing time to token-level complexity and context.
  arXiv  Detail & Related papers  (2025-10-13T21:07:05Z)
• Structured Sparse Transition Matrices to Enable State Tracking in State-Space Models [68.31088463716269]
  We propose a structured sparse parametrization of transition matrices in state-space models (SSMs)<n>Our method, PD-SSM, parametrizes the transition matrix as the product of a column one-hot matrix ($P$) and a complex-valued diagonal matrix ($D$)<n>The model significantly outperforms a wide collection of modern SSM variants on various FSA state tracking tasks.
  arXiv  Detail & Related papers  (2025-09-26T12:46:30Z)
• Scaling Up Liquid-Resistance Liquid-Capacitance Networks for Efficient Sequence Modeling [50.994194925685434]
  LrcSSM is a $textitnon-linear$ recurrent model that processes long sequences as fast as today's linear state-space layers.<n>By forcing its Jacobian matrix to be diagonal, the full sequence can be solved in parallel.<n>LrcSSM offers a formal gradient-stability guarantee that other input-varying systems such as Liquid-S4 do not provide.
  arXiv  Detail & Related papers  (2025-05-27T20:02:59Z)
• TFMQ-DM: Temporal Feature Maintenance Quantization for Diffusion Models [52.454274602380124]
  Diffusion models heavily depend on the time-step $t$ to achieve satisfactory multi-round denoising. We propose a Temporal Feature Maintenance Quantization (TFMQ) framework building upon a Temporal Information Block. Powered by the pioneering block design, we devise temporal information aware reconstruction (TIAR) and finite set calibration (FSC) to align the full-precision temporal features.
  arXiv  Detail & Related papers  (2023-11-27T12:59:52Z)
• TILDE-Q: A Transformation Invariant Loss Function for Time-Series Forecasting [8.086595073181604]
  Time-series forecasting can address real-world problems across various domains, including energy, weather, traffic, and economy. Time-series forecasting is a well-researched field, predicting complex temporal patterns such as sudden changes in sequential data still poses a challenge with current models. We propose a novel, compact loss function called TILDEQ that considers not only amplitude and phase distortions but also allows models to capture the shape of time-series sequences.
  arXiv  Detail & Related papers  (2022-10-26T21:32:20Z)
• Acting in Delayed Environments with Non-Stationary Markov Policies [57.52103323209643]
  We introduce a framework for learning and planning in MDPs where the decision-maker commits actions that are executed with a delay of $m$ steps. We prove that with execution delay, deterministic Markov policies in the original state-space are sufficient for attaining maximal reward, but need to be non-stationary. We devise a non-stationary Q-learning style model-based algorithm that solves delayed execution tasks without resorting to state-augmentation.
  arXiv  Detail & Related papers  (2021-01-28T13:35:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.