ChronoSSM: Training for Temporally Aware Representations in Autoregressive State Space Models

• URL: http://arxiv.org/abs/2608.10120v1
• Date: Mon, 10 Aug 2026 18:35:19 GMT
• Title: ChronoSSM: Training for Temporally Aware Representations in Autoregressive State Space Models
• Abstract summary: We introduce ChronoSSM, an autoregressive State Space Model (SSM) that jointly models events and timestamps with a shared backbone.<n>We show that temporal supervision can produce more temporally informative representations without materially degrading autoregressive event modeling.
• Score: 0.5366467936613011
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modern sequence models, from Transformers to State Space Models, have enabled powerful generative modeling across diverse domains, yet they are typically trained to predict what happens while treating when it happens as a secondary concern. In data-mining settings where events are associated with explicit timing information, this separation can limit temporal reasoning, anomaly detection, and faithful reconstruction of event chronology. A common strategy is to treat timing as an auxiliary signal, training a separate timing model using representations learned solely for event prediction. However, this two-stage approach implicitly assumes that representations optimized for event prediction already contain sufficient temporal structure. We introduce ChronoSSM, an autoregressive State Space Model (SSM) that jointly models events and timestamps with a shared backbone trained using combined token and temporal generation objectives. We compare the joint regime, where temporal supervision updates the backbone, with the two-stage regime, where timing is learned only using the frozen event representations. Across four domains spanning dense and partial timestamp supervision, joint training consistently makes inter-arrival information more recoverable from frozen representations without any systematic degradation in content-generation quality overall. Our results show that temporal supervision can produce more temporally informative representations without materially degrading autoregressive event modeling.

Related papers

• Generative Event Pretraining with Foundation Model Alignment [25.86429472344715]
  Event cameras provide robust visual signals under fast motion and challenging illumination conditions.<n>Their unique sensing characteristics and limited labeled data make it challenging to train event-based visual foundation models.<n>We propose GEP (Generative Event Pretraining), a two-stage framework that transfers semantic knowledge learned from internet-scale image datasets to event data.
  arXiv  Detail & Related papers  (2026-03-24T10:10:34Z)
• Stochastic Event Prediction via Temporal Motif Transitions [0.0]
  temporal link prediction methods typically frame the task as binary classification with negative sampling.<n>We introduce STEP (STochastic Event Predictor), a framework that reformulates temporal link prediction as a sequential forecasting problem in continuous time.
  arXiv  Detail & Related papers  (2026-03-06T03:51:40Z)
• UniDiff: A Unified Diffusion Framework for Multimodal Time Series Forecasting [90.47915032778366]
  We propose UniDiff, a unified diffusion framework for multimodal time series forecasting.<n>At its core lies a unified and parallel fusion module, where a single cross-attention mechanism integrates structural information from timestamps and semantic context from texts.<n>Experiments on real-world benchmark datasets across eight domains demonstrate that the proposed UniDiff model achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-12-08T05:36:14Z)
• TIMED: Adversarial and Autoregressive Refinement of Diffusion-Based Time Series Generation [0.31498833540989407]
  TIMED is a unified generative framework that captures global structure via a forward-reverse diffusion process.<n>To further align the real and synthetic distributions in feature space, TIMED incorporates a Maximum Mean Discrepancy (MMD) loss.<n>We show that TIMED generates more realistic and temporally coherent sequences than state-of-the-art generative models.
  arXiv  Detail & Related papers  (2025-09-23T23:05:40Z)
• EventTSF: Event-Aware Non-Stationary Time Series Forecasting [73.54313384419792]
  EventTSF is an autoregressive generation framework that integrates historical time series with textual events to make subsequent forecasts.<n>Experiments on 8 synthetic and real-world datasets show that EventTSF outperforms 12 baselines across diverse event-aware non-stationary time series forecasting scenarios.
  arXiv  Detail & Related papers  (2025-08-19T01:28:47Z)
• Towards Foundation Model on Temporal Knowledge Graph Reasoning [17.165969719351125]
  Temporal Knowledge Graphs (TKGs) store temporal facts with quadruple formats (s, p, o, t)<n>New model employs sinusoidal positional encodings to capture fine-grained temporal patterns.<n>PostRA demonstrates strong zero-shot performance on unseen temporal knowledge graphs.
  arXiv  Detail & Related papers  (2025-06-04T09:19:49Z)
• DualCast: A Model to Disentangle Aperiodic Events from Traffic Series [15.49210279729803]
  We propose DualCast, a dual-branch framework that disentangles traffic signals into spatial intrinsic-temporal patterns and external environmental contexts.<n>We use a cross-time attention mechanism to capture high-order spatial-temporal relationships from both periodic and aperiodic patterns.<n>We integrate it with recent traffic forecasting models, consistently reducing their forecasting errors by up to 9.6% on multiple real datasets.
  arXiv  Detail & Related papers  (2024-11-27T12:17:50Z)
• TimeSiam: A Pre-Training Framework for Siamese Time-Series Modeling [67.02157180089573]
  Time series pre-training has recently garnered wide attention for its potential to reduce labeling expenses and benefit various downstream tasks. This paper proposes TimeSiam as a simple but effective self-supervised pre-training framework for Time series based on Siamese networks.
  arXiv  Detail & Related papers  (2024-02-04T13:10:51Z)
• Continuous-time convolutions model of event sequences [46.3471121117337]
  Event sequences are non-uniform and sparse, making traditional models unsuitable. We propose COTIC, a method based on an efficient convolution neural network designed to handle the non-uniform occurrence of events over time. COTIC outperforms existing models in predicting the next event time and type, achieving an average rank of 1.5 compared to 3.714 for the nearest competitor.
  arXiv  Detail & Related papers  (2023-02-13T10:34:51Z)
• Persistence-Based Discretization for Learning Discrete Event Systems from Time Series [50.591267188664666]
  Persist is a discretization method that intends to create persisting symbols by using a score called persistence score. We replace the metric used in the persistence score, the Kullback-Leibler divergence, with the Wasserstein distance. Experiments show that the improved persistence score enhances Persist's ability to capture the information of the original time series.
  arXiv  Detail & Related papers  (2023-01-12T14:10:30Z)
• Effective Distant Supervision for Temporal Relation Extraction [49.20329405920023]
  A principal barrier to training temporal relation extraction models in new domains is the lack of varied, high quality examples. We present a method of automatically collecting distantly-supervised examples of temporal relations.
  arXiv  Detail & Related papers  (2020-10-24T03:17:31Z)
• Unsupervised Video Decomposition using Spatio-temporal Iterative Inference [31.97227651679233]
  Multi-object scene decomposition is a fast-emerging problem in learning. We show that our model has a high accuracy even without color information. We demonstrate the decomposition, segmentation prediction capabilities of our model and show that it outperforms the state-of-the-art on several benchmark datasets.
  arXiv  Detail & Related papers  (2020-06-25T22:57:17Z)
• Predicting Temporal Sets with Deep Neural Networks [50.53727580527024]
  We propose an integrated solution based on the deep neural networks for temporal sets prediction. A unique perspective is to learn element relationship by constructing set-level co-occurrence graph. We design an attention-based module to adaptively learn the temporal dependency of elements and sets.
  arXiv  Detail & Related papers  (2020-06-20T03:29:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.