Can Multimodal LLMs Perform Time Series Anomaly Detection?

• URL: http://arxiv.org/abs/2502.17812v1
• Date: Tue, 25 Feb 2025 03:37:43 GMT
• Title: Can Multimodal LLMs Perform Time Series Anomaly Detection?
• Authors: Xiongxiao Xu, Haoran Wang, Yueqing Liang, Philip S. Yu, Yue Zhao, Kai Shu,
• Abstract summary: We propose VisualTimeAnomaly benchmark to evaluate MLLMs in time series anomaly detection (TSAD)<n>Our approach transforms time series numerical data into the image format and feed these images into various MLLMs.<n>In total, VisualTimeAnomaly contains 12.4k time series images spanning 3 scenarios and 3 anomaly granularities with 9 anomaly types across 8 MLLMs.
• Score: 55.534264764673296
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) have been increasingly used in time series analysis. However, the potential of multimodal LLMs (MLLMs), particularly vision-language models, for time series remains largely under-explored. One natural way for humans to detect time series anomalies is through visualization and textual description. Motivated by this, we raise a critical and practical research question: Can multimodal LLMs perform time series anomaly detection? To answer this, we propose VisualTimeAnomaly benchmark to evaluate MLLMs in time series anomaly detection (TSAD). Our approach transforms time series numerical data into the image format and feed these images into various MLLMs, including proprietary models (GPT-4o and Gemini-1.5) and open-source models (LLaVA-NeXT and Qwen2-VL), each with one larger and one smaller variant. In total, VisualTimeAnomaly contains 12.4k time series images spanning 3 scenarios and 3 anomaly granularities with 9 anomaly types across 8 MLLMs. Starting with the univariate case (point- and range-wise anomalies), we extend our evaluation to more practical scenarios, including multivariate and irregular time series scenarios, and variate-wise anomalies. Our study reveals several key insights: 1) MLLMs detect range- and variate-wise anomalies more effectively than point-wise anomalies. 2) MLLMs are highly robust to irregular time series, even with 25% of the data missing. 3) Open-source MLLMs perform comparably to proprietary models in TSAD. While open-source MLLMs excel on univariate time series, proprietary MLLMs demonstrate superior effectiveness on multivariate time series. To the best of our knowledge, this is the first work to comprehensively investigate MLLMs for TSAD, particularly for multivariate and irregular time series scenarios. We release our dataset and code at https://github.com/mllm-ts/VisualTimeAnomaly to support future research.

Related papers

• Visual Chronicles: Using Multimodal LLMs to Analyze Massive Collections of Images [58.38037252899024]
  We present a system using Multimodal LLMs to analyze a large database with tens of millions of images. We aim to capture frequent co-occurring changes ("trends") across a city over a certain period. We find it significantly outperforms baselines and is able to discover interesting trends from images captured in large cities.
  arXiv  Detail & Related papers  (2025-04-11T17:55:45Z)
• LLM-PS: Empowering Large Language Models for Time Series Forecasting with Temporal Patterns and Semantics [56.99021951927683]
  Time Series Forecasting (TSF) is critical in many real-world domains like financial planning and health monitoring. Existing Large Language Models (LLMs) usually perform suboptimally because they neglect the inherent characteristics of time series data. We propose LLM-PS to empower the LLM for TSF by learning the fundamental textitPatterns and meaningful textitSemantics from time series data.
  arXiv  Detail & Related papers  (2025-03-12T11:45:11Z)
• Position: Empowering Time Series Reasoning with Multimodal LLMs [49.73647759532127]
  We argue that multimodal language models (MLLMs) can enable more powerful and flexible reasoning for time series analysis.<n>We call on researchers and practitioners to leverage this potential by developing strategies that prioritize trust, interpretability, and robust reasoning in MLLMs.
  arXiv  Detail & Related papers  (2025-02-03T16:10:48Z)
• Can Multimodal LLMs do Visual Temporal Understanding and Reasoning? The answer is No! [22.75945626401567]
  We propose a challenging evaluation benchmark named TemporalVQA.<n>The first part requires MLLMs to determine the sequence of events by analyzing temporally consecutive video frames.<n>The second part presents image pairs with varying time differences, framed as multiple-choice questions, asking MLLMs to estimate the time-lapse between images with options ranging from seconds to years.<n>Our evaluations of advanced MLLMs, including models like GPT-4o and Gemini-1.5-Pro, reveal significant challenges.
  arXiv  Detail & Related papers  (2025-01-18T06:41:48Z)
• Can LLMs Understand Time Series Anomalies? [20.848375315326305]
  Large Language Models (LLMs) have gained popularity in time series forecasting, but their potential for anomaly detection remains largely unexplored. Our study investigates whether LLMs can understand and detect anomalies in time series data, focusing on zero-shot and few-shot scenarios. Our results suggest that while LLMs can understand time series anomalies, many common conjectures based on their reasoning capabilities do not hold.
  arXiv  Detail & Related papers  (2024-10-07T19:16:02Z)
• Towards Time Series Reasoning with LLMs [0.4369058206183195]
  We propose a novel multi-modal time-series LLM approach that learns generalizable information across various domains with powerful zero-shot performance. We show that our model learns a latent representation that reflects specific time-series features, as well as outperforming GPT-4o on a set of zero-shot reasoning tasks.
  arXiv  Detail & Related papers  (2024-09-17T17:23:44Z)
• Can LLMs Serve As Time Series Anomaly Detectors? [33.28502093260832]
  An emerging topic in large language models (LLMs) is their application to time series forecasting. In this paper, we investigate the capabilities of LLMs, specifically GPT-4 and LLaMA3, in detecting and explaining anomalies in time series.
  arXiv  Detail & Related papers  (2024-08-06T23:14:39Z)
• MIBench: Evaluating Multimodal Large Language Models over Multiple Images [70.44423964171088]
  We propose a new benchmark MIBench, to comprehensively evaluate fine-grained abilities of MLLMs in multi-image scenarios. Specifically, MIBench categorizes the multi-image abilities into three scenarios: multi-image instruction (MII), multimodal knowledge-seeking (MKS) and multimodal in-context learning (MIC) The results reveal that although current models excel in single-image tasks, they exhibit significant shortcomings when faced with multi-image inputs.
  arXiv  Detail & Related papers  (2024-07-21T21:22:58Z)
• MultiCast: Zero-Shot Multivariate Time Series Forecasting Using LLMs [0.8329456268842227]
  MultiCast is a zero-shot LLM-based approach for multivariate time series forecasting. Three novel token multiplexing solutions effectively reduce dimensionality while preserving key repetitive patterns. We showcase the performance of our approach in terms of RMSE and execution time against state-of-the-art approaches on three real-world datasets.
  arXiv  Detail & Related papers  (2024-05-23T16:16:00Z)
• Graph Spatiotemporal Process for Multivariate Time Series Anomaly Detection with Missing Values [67.76168547245237]
  We introduce a novel framework called GST-Pro, which utilizes a graphtemporal process and anomaly scorer to detect anomalies. Our experimental results show that the GST-Pro method can effectively detect anomalies in time series data and outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-01-11T10:10:16Z)
• Time-LLM: Time Series Forecasting by Reprogramming Large Language Models [110.20279343734548]
  Time series forecasting holds significant importance in many real-world dynamic systems. We present Time-LLM, a reprogramming framework to repurpose large language models for time series forecasting. Time-LLM is a powerful time series learner that outperforms state-of-the-art, specialized forecasting models.
  arXiv  Detail & Related papers  (2023-10-03T01:31:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.