A Cost-Effective Multimodal LLM Reasoning Framework for Question Answering over Irregular Clinical Time Series

• URL: http://arxiv.org/abs/2607.25947v1
• Date: Tue, 28 Jul 2026 16:33:41 GMT
• Title: A Cost-Effective Multimodal LLM Reasoning Framework for Question Answering over Irregular Clinical Time Series
• Abstract summary: Question answering over irregular clinical time series (ICTS) plays a pivotal role in a wide range of healthcare applications.<n>Recent multimodal time-series large language models (LLMs) have shown considerable promise in general-purpose time-series QA.<n>We propose ClinPRISM, a cost-effective multimodal LLM reasoning framework for question answering over ICTS data.
• Score: 10.789471760777564
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Question answering (QA) over irregular clinical time series (ICTS) plays a pivotal role in a wide range of healthcare applications. Although recent multimodal time-series large language models (LLMs) have shown considerable promise in general-purpose time-series QA, they remain poorly equipped to model the sparsity, asynchrony, and irregular sampling patterns of clinical observations. To fill this gap, we propose ClinPRISM, a cost-effective multimodal LLM reasoning framework for question answering over ICTS data. First, we devise an irregularity-aware multi-scale encoder to capture sparse clinical evidence at diverse temporal scales. Then, we propose a temporal evidence distiller to integrate representations across these scales and compress them into a small number of LLM-compatible tokens. Moreover, we introduce a progressive alignment strategy that sequentially aligns the irregular trajectories with the LLM's textual embedding space. To facilitate training, we construct 30,000 clinical time series paired with multi-scale descriptions, together with 41,000 instruction-tuning instances spanning 11 tasks. Using a 4-billion-parameter LLM backbone, ClinPRISM achieves state-of-the-art performance on the held-out evaluation benchmark while using only 16 time-series tokens and achieving an average inference latency of 0.15 seconds per question.

Related papers

• MMTClinic: Multimodal, Multilingual Time Series Question Answering and Reasoning Benchmark for Clinical Domain [17.198422651795912]
  MMTClinic is a benchmark designed to evaluate large language models on complex reasoning and question-answering tasks involving clinical time-series.<n>It includes 30,000 QA pairs (15,000 multiple choice questions (MCQs) and 15,000 open-ended questions) across five languages: English, Hindi, Bengali, Marathi, and Tamil.<n>We evaluate 13 state-of-the-art LLMs in zero-shot, few-shot, and chain-of-thought settings.
  arXiv  Detail & Related papers  (2026-09-04T07:59:19Z)
• From Recognition to Understanding: Unlocking Cognitive Time Series Reasoning with LLMs [21.316087276322214]
  Time series analysis has recently been coupled with Large Language Models (LLMs) to leverage their reasoning and world knowledge capabilities.<n>We introduce TSCognition, a multimodal benchmark for multi-dimensional time series reasoning.<n>We propose TSAlign, a unified framework that encodes time series into compact patch-level representations.
  arXiv  Detail & Related papers  (2026-06-20T16:08:44Z)
• TSAQA: Time Series Analysis Question And Answering Benchmark [85.35545785252309]
  Time series data are integral to critical applications across domains such as finance, healthcare, transportation, and environmental science.<n>We introduce TSAQA, a novel unified benchmark designed to broaden task coverage and evaluate diverse temporal analysis capabilities.
  arXiv  Detail & Related papers  (2026-01-30T17:28:56Z)
• Parallel Vision Token Scheduling for Fast and Accurate Multimodal LMMs Inference [68.4758228017823]
  ParVTS partitions visual tokens into subject and non-subject groups, processes them in parallel to transfer their semantics into question tokens, and discards the non-subject path mid-inference.<n>Experiments show that ParVTS prunes up to 88.9% of visual tokens with minimal performance drop, achieving 1.77x speedup and 70% FLOPs reduction.
  arXiv  Detail & Related papers  (2025-11-24T08:29:36Z)
• SPEAR: Soft Prompt Enhanced Anomaly Recognition for Time Series Data [15.670885465827206]
  Time series anomaly detection plays a crucial role in a wide range of fields, such as healthcare and internet traffic monitoring.<n>The emergence of large language models (LLMs) offers new opportunities for detecting anomalies in the ubiquitous time series data.<n>We propose Soft Prompt Enhanced Anomaly Recognition (SPEAR), a novel approach to leverage LLMs for anomaly detection with soft prompts and quantization.
  arXiv  Detail & Related papers  (2025-10-04T22:20:48Z)
• SciTS: Scientific Time Series Understanding and Generation with LLMs [53.35994674187729]
  We introduce SciTS, a benchmark spanning 12 scientific domains and 43 tasks.<n>We benchmark 17 models, including text-only LLMs, multimodal LLMs, and unified time series models.<n>We then introduce Time Omni, a framework that equips LLMs with the ability to understand and generate time series.
  arXiv  Detail & Related papers  (2025-09-26T09:25:16Z)
• Time-RA: Towards Time Series Reasoning for Anomaly with LLM Feedback [55.284574165467525]
  Time-series Reasoning for Anomaly (Time-RA) transforms classical time series anomaly detection into a generative, reasoning-intensive task.<n>Also, we introduce the first real-world multimodal benchmark dataset, RATs40K, explicitly annotated for anomaly reasoning.
  arXiv  Detail & Related papers  (2025-07-20T18:02:50Z)
• LLM-PS: Empowering Large Language Models for Time Series Forecasting with Temporal Patterns and Semantics [56.99021951927683]
  Time Series Forecasting (TSF) is critical in many real-world domains like financial planning and health monitoring.<n>Existing Large Language Models (LLMs) usually perform suboptimally because they neglect the inherent characteristics of time series data.<n>We propose LLM-PS to empower the LLM for TSF by learning the fundamental textitPatterns and meaningful textitSemantics from time series data.
  arXiv  Detail & Related papers  (2025-03-12T11:45:11Z)
• Large Language Models are Few-shot Multivariate Time Series Classifiers [23.045734479292356]
  Large Language Models (LLMs) have been extensively applied in time series analysis.<n>Yet, their utility in the few-shot classification (i.e., a crucial training scenario) is underexplored.<n>We aim to leverage the extensive pre-trained knowledge in LLMs to overcome the data scarcity problem.
  arXiv  Detail & Related papers  (2025-01-30T03:59:59Z)
• ChatTS: Aligning Time Series with LLMs via Synthetic Data for Enhanced Understanding and Reasoning [10.854285913078257]
  This paper introduces ChatTS, a novel MLLM designed for time series analysis.<n>ChatTS treats time series as a modality, similar to how vision MLLMs process images.<n>Time Series Evol-Instruct generates diverse time series Q&As, enhancing the model's reasoning capabilities.
  arXiv  Detail & Related papers  (2024-12-04T08:06:15Z)
• TableTime: Reformulating Time Series Classification as Training-Free Table Understanding with Large Language Models [20.077169400691435]
  Large language models (LLMs) have demonstrated their effectiveness in multivariate time series classification.<n>LLMs directly encode embeddings for time series within the latent space of LLMs from scratch to align with semantic space of LLMs.<n>We propose TableTime, which reformulates MTSC as a table understanding task.
  arXiv  Detail & Related papers  (2024-11-24T07:02:32Z)
• Hierarchical Multimodal LLMs with Semantic Space Alignment for Enhanced Time Series Classification [4.5939667818289385]
  HiTime is a hierarchical multi-modal model that seamlessly integrates temporal information into large language models. Our findings highlight the potential of integrating temporal features into LLMs, paving the way for advanced time series analysis.
  arXiv  Detail & Related papers  (2024-10-24T12:32:19Z)
• Time-LLM: Time Series Forecasting by Reprogramming Large Language Models [110.20279343734548]
  Time series forecasting holds significant importance in many real-world dynamic systems. We present Time-LLM, a reprogramming framework to repurpose large language models for time series forecasting. Time-LLM is a powerful time series learner that outperforms state-of-the-art, specialized forecasting models.
  arXiv  Detail & Related papers  (2023-10-03T01:31:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.