ChatTime: A Unified Multimodal Time Series Foundation Model Bridging Numerical and Textual Data

• URL: http://arxiv.org/abs/2412.11376v1
• Date: Mon, 16 Dec 2024 02:04:06 GMT
• Title: ChatTime: A Unified Multimodal Time Series Foundation Model Bridging Numerical and Textual Data
• Authors: Chengsen Wang, Qi Qi, Jingyu Wang, Haifeng Sun, Zirui Zhuang, Jinming Wu, Lei Zhang, Jianxin Liao,
• Abstract summary: ChatTime is a unified framework for time series and text processing.<n>As an out-of-the-box multimodal time series foundation model, ChatTime provides zero-shot forecasting capability.<n>We design a series of experiments to verify the superior performance of ChatTime across multiple tasks and scenarios.
• Score: 26.300515935897415
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Human experts typically integrate numerical and textual multimodal information to analyze time series. However, most traditional deep learning predictors rely solely on unimodal numerical data, using a fixed-length window for training and prediction on a single dataset, and cannot adapt to different scenarios. The powered pre-trained large language model has introduced new opportunities for time series analysis. Yet, existing methods are either inefficient in training, incapable of handling textual information, or lack zero-shot forecasting capability. In this paper, we innovatively model time series as a foreign language and construct ChatTime, a unified framework for time series and text processing. As an out-of-the-box multimodal time series foundation model, ChatTime provides zero-shot forecasting capability and supports bimodal input/output for both time series and text. We design a series of experiments to verify the superior performance of ChatTime across multiple tasks and scenarios, and create four multimodal datasets to address data gaps. The experimental results demonstrate the potential and utility of ChatTime.

Related papers

• Chat-TS: Enhancing Multi-Modal Reasoning Over Time-Series and Natural Language Data [22.274663165215237]
  Time-series analysis is critical for a wide range of fields such as healthcare, finance, transportation, and energy. Current time-series models are limited in their ability to perform reasoning that involves both time-series and their textual content. Chat-TS integrates time-series tokens into LLMs' vocabulary, enhancing its reasoning ability over both modalities.
  arXiv  Detail & Related papers  (2025-03-13T21:05:11Z)
• TimesBERT: A BERT-Style Foundation Model for Time Series Understanding [72.64824086839631]
  GPT-style models have been positioned as foundation models for time series forecasting. BERT-style architecture has not been fully unlocked for time series understanding. We design TimesBERT to learn generic representations of time series. Our model is pre-trained on 260 billion time points across diverse domains.
  arXiv  Detail & Related papers  (2025-02-28T17:14:44Z)
• TimeCAP: Learning to Contextualize, Augment, and Predict Time Series Events with Large Language Model Agents [52.13094810313054]
  TimeCAP is a time-series processing framework that creatively employs Large Language Models (LLMs) as contextualizers of time series data. TimeCAP incorporates two independent LLM agents: one generates a textual summary capturing the context of the time series, while the other uses this enriched summary to make more informed predictions. Experimental results on real-world datasets demonstrate that TimeCAP outperforms state-of-the-art methods for time series event prediction.
  arXiv  Detail & Related papers  (2025-02-17T04:17:27Z)
• Language in the Flow of Time: Time-Series-Paired Texts Weaved into a Unified Temporal Narrative [65.84249211767921]
  Texts as Time Series (TaTS) considers the time-series-paired texts to be auxiliary variables of the time series. TaTS can be plugged into any existing numerical-only time series models and enable them to handle time series data with paired texts effectively.
  arXiv  Detail & Related papers  (2025-02-13T03:43:27Z)
• Unveiling the Potential of Text in High-Dimensional Time Series Forecasting [12.707274099874384]
  We propose a novel framework that integrates time series models with Large Language Models. Inspired by multimodal models, our method combines time series and textual data in the dual-tower structure. Experiments demonstrate that incorporating text enhances high-dimensional time series forecasting performance.
  arXiv  Detail & Related papers  (2025-01-13T04:10:45Z)
• Multi-Modal Forecaster: Jointly Predicting Time Series and Textual Data [23.10730301634422]
  Current forecasting approaches are largely unimodal and ignore the rich textual data that often accompany the time series. We develop TimeText Corpus (TTC), a carefully curated, time-aligned text and time dataset for multimodal forecasting. Our dataset is composed of sequences of numbers and text aligned to timestamps, and includes data from two different domains: climate science and healthcare.
  arXiv  Detail & Related papers  (2024-11-11T06:04:15Z)
• Text2Freq: Learning Series Patterns from Text via Frequency Domain [8.922661807801227]
  Text2Freq is a cross-modality model that integrates text and time series data via the frequency domain. Our experiments on paired datasets of real-world stock prices and synthetic texts show that Text2Freq achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2024-11-01T16:11:02Z)
• Context is Key: A Benchmark for Forecasting with Essential Textual Information [87.3175915185287]
  "Context is Key" (CiK) is a time series forecasting benchmark that pairs numerical data with diverse types of carefully crafted textual context. We evaluate a range of approaches, including statistical models, time series foundation models, and LLM-based forecasters. Our experiments highlight the importance of incorporating contextual information, demonstrate surprising performance when using LLM-based forecasting models, and also reveal some of their critical shortcomings.
  arXiv  Detail & Related papers  (2024-10-24T17:56:08Z)
• Large Language Models Are Zero-Shot Time Series Forecasters [48.73953666153385]
  By encoding time series as a string of numerical digits, we can frame time series forecasting as next-token prediction in text. We find that large language models (LLMs) such as GPT-3 and LLaMA-2 can surprisingly zero-shot extrapolate time series at a level comparable to or exceeding the performance of purpose-built time series models trained on the downstream tasks.
  arXiv  Detail & Related papers  (2023-10-11T19:01:28Z)
• Time-LLM: Time Series Forecasting by Reprogramming Large Language Models [110.20279343734548]
  Time series forecasting holds significant importance in many real-world dynamic systems. We present Time-LLM, a reprogramming framework to repurpose large language models for time series forecasting. Time-LLM is a powerful time series learner that outperforms state-of-the-art, specialized forecasting models.
  arXiv  Detail & Related papers  (2023-10-03T01:31:25Z)
• Human-like Time Series Summaries via Trend Utility Estimation [13.560018516096754]
  We propose a model to create human-like text descriptions for time series. Our system finds patterns in time series data and ranks these patterns based on empirical observations of human behavior. The output of our system is a natural language description of time series that attempts to match a human's summary of the same data.
  arXiv  Detail & Related papers  (2020-01-16T06:09:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.