Liquid Gated Attention

• URL: http://arxiv.org/abs/2608.30695v1
• Date: Mon, 31 Aug 2026 12:30:09 GMT
• Title: Liquid Gated Attention
• Abstract summary: Liquid Gated Attention (LGA) is a solver-free parallel temporal operator.<n>Building on LGA, we instantiate LFormer, a modular backbone for continuous-time representation learning.
• Score: 12.636376869841826
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Real-world time series often exhibit irregular sampling and extended temporal horizons, requiring models to capture continuous-time dynamics across arbitrary intervals without prohibitive scaling costs. Discrete-time methods collapse variable time intervals into static positional steps; solver-dependent continuous-time models preserve temporal structure but rely on sequential integration, precluding parallelization; and solver-free approximations avoid this cost yet none couples observed time intervals with input-driven state modulation. We propose Liquid Gated Attention (LGA), a solver-free parallel temporal operator. By parameterizing an input-driven gating mechanism with observed time intervals, LGA introduces a continuous-time inductive bias and formulates hidden state evolution as a fast-weight associative memory, enabling parallel computation across the temporal dimension. Using matrix associativity in non-causal encoding and a prefix scan in causal encoding, LGA attains linear temporal complexity in sequence length in both modes. A sequence-level normalization bounds cumulative temporal decay for stable long-horizon optimization. Building on LGA, we instantiate LFormer, a modular backbone for continuous-time representation learning. Across six tasks and sixteen datasets spanning up to 17,984 steps, LFormer demonstrates long-range dependency modeling, fine-grained state tracking, and trajectory reconstruction from sparse and noisy observations, while delivering competitive performance against state-of-the-art discrete-time and continuous-time baselines with linear scaling efficiency.

Related papers

• Faithful Embeddings of Irregular and Asynchronous Data for Online Log-NCDEs [43.7379426352196]
  Interpolation- and imputation-based embeddings reconstruct a continuous observation path.<n>We show that this reconstruction step is unnecessary under mild conditions.<n>We introduce a continuous and injective embedding for Log-NCDEs, a universal class of continuous-time models.
  arXiv  Detail & Related papers  (2026-05-28T16:46:11Z)
• Latent Laplace Diffusion for Irregular Multivariate Time Series [16.474933482743513]
  We present Latent Laplace Diffusion, a generative framework that models the target as a low-dimensional latent trajectory.<n>LLapDiff enables horizon-wide generation without step-by-step integration over physical time.<n>We also link continuous dynamics to irregular observations through renewal-averaging analysis.
  arXiv  Detail & Related papers  (2026-05-19T13:04:53Z)
• Continuous-Time Distribution Matching for Few-Step Diffusion Distillation [57.28746398500951]
  We introduce Continuous-Time Distribution Matching (CDM), migrating the DMD framework from discrete anchoring to continuous optimization.<n>CDM achieves this through two continuous-time designs. First, we replace the fixed discrete schedule with a dynamic continuous schedule of random length.<n>Second, we propose a continuous-time alignment objective that performs active off-trajectory matching on latents extrapolated via the student's velocity field.
  arXiv  Detail & Related papers  (2026-05-07T14:56:39Z)
• FoSS: Modeling Long Range Dependencies and Multimodal Uncertainty in Trajectory Prediction via Fourier State Space Integration [21.39395366378851]
  We present FoSS, a dual-branch framework that unifies frequency-domain reasoning with linear-time sequence modeling.<n>Experiments on Argoverse 1 and Argoverse 2 benchmarks demonstrate that FoSS achieves state-of-the-art accuracy while reducing computation by 22.5% and parameters by over 40%.
  arXiv  Detail & Related papers  (2026-03-01T21:38:59Z)
• Forecasting as Rendering: A 2D Gaussian Splatting Framework for Time Series Forecasting [79.37674445572462]
  Time series forecasting (TSF) remains a challenging problem due to the intricate entanglement of intraperiod-fluctuations and interperiod-trends.<n>Treating reshaped tensors as static images results in a topological mismatch.<n> relying on uniform fixed-size representations allocates modeling capacity inefficiently.<n>TimeGS is a novel framework that fundamentally shifts the forecasting paradigm from regression to 2D generative rendering.
  arXiv  Detail & Related papers  (2026-02-10T14:13:36Z)
• Less Is More: Generating Time Series with LLaMA-Style Autoregression in Simple Factorized Latent Spaces [29.349850914912846]
  We propose FAR-TS, a simple yet effective framework that combines disentangled factorization with an autoregressive Transformer over a discrete, quantized latent space to generate time series.<n>A LLaMA-style autoregressive Transformer then models these token sequences, enabling fast and controllable generation of sequences with arbitrary length.
  arXiv  Detail & Related papers  (2025-11-07T04:15:38Z)
• Oscillatory State-Space Models [61.923849241099184]
  We propose Lineary State-Space models (LinOSS) for efficiently learning on long sequences.<n>A stable discretization, integrated over time using fast associative parallel scans, yields the proposed state-space model.<n>We show that LinOSS is universal, i.e., it can approximate any continuous and causal operator mapping between time-varying functions.
  arXiv  Detail & Related papers  (2024-10-04T22:00:13Z)
• Convolutional State Space Models for Long-Range Spatiotemporal Modeling [65.0993000439043]
  ConvS5 is an efficient variant for long-rangetemporal modeling. It significantly outperforms Transformers and ConvNISTTM on a long horizon Moving-Lab experiment while training 3X faster than ConvLSTM and generating samples 400X faster than Transformers.
  arXiv  Detail & Related papers  (2023-10-30T16:11:06Z)
• Deep Explicit Duration Switching Models for Time Series [84.33678003781908]
  We propose a flexible model that is capable of identifying both state- and time-dependent switching dynamics. State-dependent switching is enabled by a recurrent state-to-switch connection. An explicit duration count variable is used to improve the time-dependent switching behavior.
  arXiv  Detail & Related papers  (2021-10-26T17:35:21Z)
• The Connection between Discrete- and Continuous-Time Descriptions of Gaussian Continuous Processes [60.35125735474386]
  We show that discretizations yielding consistent estimators have the property of invariance under coarse-graining' This result explains why combining differencing schemes for derivatives reconstruction and local-in-time inference approaches does not work for time series analysis of second or higher order differential equations.
  arXiv  Detail & Related papers  (2021-01-16T17:11:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.