Chronicle: A Multimodal Foundation Model for Joint Language and Time Series Understanding

• URL: http://arxiv.org/abs/2605.20268v1
• Date: Mon, 18 May 2026 21:39:05 GMT
• Title: Chronicle: A Multimodal Foundation Model for Joint Language and Time Series Understanding
• Abstract summary: We present Chronicle, a compact 324M- parameter decoder-only transformer trained from scratch on natural language and time series.<n>To our knowledge, Chronicle is the first model jointly pretrained on text and time series from scratch, and the first multimodal model evaluated against dedicated foundation models in both domains.
• Score: 11.063684308771471
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Real-world time series come with text: metadata, descriptions, news, reports. Yet time series foundation models process numerical sequences in isolation, and the multimodal text-and-time-series models that attempt to bridge the two all adapt a pretrained language model post hoc, inheriting representations shaped without ever seeing temporal data. These models are also evaluated almost exclusively against other multimodal baselines, not against the strongest unimodal foundation models in either domain, leaving open whether joint training is needed at all. We present Chronicle, a compact 324M-parameter decoder-only transformer trained from scratch on natural language and time series within a single unified architecture. Both modalities share the same transformer blocks, attention mechanism, and residual stream; the bulk of pretraining uses unimodal batches so cross-modal capability emerges purely from shared parameters, with a short alignment stage that interleaves the two. To our knowledge, Chronicle is the first model jointly pretrained on text and time series from scratch, and the first multimodal model evaluated against dedicated foundation models in both domains. It matches Gemma-3-270M-PT on 19 NLU tasks, sets a new bar for frozen-embedding time series classification on 24 UCR/UEA datasets, and produces multimodal forecasts on Time-MMD that beat every supervised fusion baseline, all from a single backbone.

Related papers

• Empowering Time Series Analysis with Large-Scale Multimodal Pretraining [34.22079919081765]
  Building multimodal foundation models is a natural next step, but it faces key challenges.<n>Lack of a unified multimodal pretraining paradigm and large-scale multimodal corpora for time series analysis.<n>How to effectively integrate heterogeneous modalities and enhance model generalization.
  arXiv  Detail & Related papers  (2026-02-05T13:26:35Z)
• UniDiff: A Unified Diffusion Framework for Multimodal Time Series Forecasting [90.47915032778366]
  We propose UniDiff, a unified diffusion framework for multimodal time series forecasting.<n>At its core lies a unified and parallel fusion module, where a single cross-attention mechanism integrates structural information from timestamps and semantic context from texts.<n>Experiments on real-world benchmark datasets across eight domains demonstrate that the proposed UniDiff model achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-12-08T05:36:14Z)
• BEDTime: A Unified Benchmark for Automatically Describing Time Series [8.466823017204641]
  We argue that successful multi-modal models should be able to recognize, differentiate, and generate language descriptions of time series.<n>We then create BEDTime, the first benchmark dataset to assess models on each task.<n>Using BEDTime, we evaluate 13 state-of-the-art models, and find that dedicated time series foundation models severely underperform.
  arXiv  Detail & Related papers  (2025-09-05T16:18:20Z)
• MoTM: Towards a Foundation Model for Time Series Imputation based on Continuous Modeling [0.764671395172401]
  We propose a first step to fill a gap by leveraging implicit neural representations (INRs)<n>MoTM combines a basis of INRs, each trained independently on a distinct family of time series, with a ridge regressor that adapts to the observed context at inference.<n>We demonstrate robust in-domain and out-of-domain generalization across diverse imputation scenarios.
  arXiv  Detail & Related papers  (2025-07-17T15:16:30Z)
• ChronoSteer: Bridging Large Language Model and Time Series Foundation Model via Synthetic Data [22.81326423408988]
  We introduce ChronoSteer, a multimodal TSFM that can be steered through textual revision instructions.<n>To mitigate the shortage of cross-modal instruction-series paired data, we devise a two-stage training strategy based on synthetic data.<n> ChronoSteer achieves a 25.7% improvement in prediction accuracy compared to the unimodal backbone and a 22.5% gain over the previous state-of-the-art multimodal method.
  arXiv  Detail & Related papers  (2025-05-15T08:37:23Z)
• TimesBERT: A BERT-Style Foundation Model for Time Series Understanding [72.64824086839631]
  GPT-style models have been positioned as foundation models for time series forecasting.<n>BERT-style architecture has not been fully unlocked for time series understanding.<n>We design TimesBERT to learn generic representations of time series.<n>Our model is pre-trained on 260 billion time points across diverse domains.
  arXiv  Detail & Related papers  (2025-02-28T17:14:44Z)
• Language in the Flow of Time: Time-Series-Paired Texts Weaved into a Unified Temporal Narrative [65.84249211767921]
  Texts as Time Series (TaTS) can be plugged into any existing numerical-only time series models.<n>We show that TaTS can enhance predictive performance without modifying model architectures.
  arXiv  Detail & Related papers  (2025-02-13T03:43:27Z)
• Moirai-MoE: Empowering Time Series Foundation Models with Sparse Mixture of Experts [103.725112190618]
  This paper introduces Moirai-MoE, using a single input/output projection layer while delegating the modeling of diverse time series patterns to the sparse mixture of experts. Extensive experiments on 39 datasets demonstrate the superiority of Moirai-MoE over existing foundation models in both in-distribution and zero-shot scenarios.
  arXiv  Detail & Related papers  (2024-10-14T13:01:11Z)
• UniTST: Effectively Modeling Inter-Series and Intra-Series Dependencies for Multivariate Time Series Forecasting [98.12558945781693]
  We propose a transformer-based model UniTST containing a unified attention mechanism on the flattened patch tokens. Although our proposed model employs a simple architecture, it offers compelling performance as shown in our experiments on several datasets for time series forecasting.
  arXiv  Detail & Related papers  (2024-06-07T14:39:28Z)
• Sports-Traj: A Unified Trajectory Generation Model for Multi-Agent Movement in Sports [53.637837706712794]
  We propose a Unified Trajectory Generation model, UniTraj, that processes arbitrary trajectories as masked inputs.<n>Specifically, we introduce a Ghost Spatial Masking (GSM) module, embedded within a Transformer encoder, for spatial feature extraction.<n>We benchmark three practical sports datasets, Basketball-U, Football-U, and Soccer-U, for evaluation.
  arXiv  Detail & Related papers  (2024-05-27T22:15:23Z)
• UniTS: A Unified Multi-Task Time Series Model [31.675845788410246]
  UniTS is a unified multi-task time series model that integrates predictive and generative tasks into a single framework. UniTS is tested on 38 datasets across human activity sensors, healthcare, engineering, and finance.
  arXiv  Detail & Related papers  (2024-02-29T21:25:58Z)
• Unified Training of Universal Time Series Forecasting Transformers [104.56318980466742]
  We present a Masked-based Universal Time Series Forecasting Transformer (Moirai) Moirai is trained on our newly introduced Large-scale Open Time Series Archive (LOTSA) featuring over 27B observations across nine domains. Moirai achieves competitive or superior performance as a zero-shot forecaster when compared to full-shot models.
  arXiv  Detail & Related papers  (2024-02-04T20:00:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.