Period-LLM: Extending the Periodic Capability of Multimodal Large Language Model

• URL: http://arxiv.org/abs/2505.24476v1
• Date: Fri, 30 May 2025 11:23:21 GMT
• Title: Period-LLM: Extending the Periodic Capability of Multimodal Large Language Model
• Abstract summary: Current Multimodal Large Language Models (MLLMs) struggle with periodic tasks due to limitations in: 1) lack of temporal modelling and 2) conflict between short and long periods.<n>This paper introduces Period-LLM, a multimodal large language model designed to enhance the performance of periodic tasks across various modalities.
• Score: 26.655013761142758
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Periodic or quasi-periodic phenomena reveal intrinsic characteristics in various natural processes, such as weather patterns, movement behaviors, traffic flows, and biological signals. Given that these phenomena span multiple modalities, the capabilities of Multimodal Large Language Models (MLLMs) offer promising potential to effectively capture and understand their complex nature. However, current MLLMs struggle with periodic tasks due to limitations in: 1) lack of temporal modelling and 2) conflict between short and long periods. This paper introduces Period-LLM, a multimodal large language model designed to enhance the performance of periodic tasks across various modalities, and constructs a benchmark of various difficulty for evaluating the cross-modal periodic capabilities of large models. Specially, We adopt an "Easy to Hard Generalization" paradigm, starting with relatively simple text-based tasks and progressing to more complex visual and multimodal tasks, ensuring that the model gradually builds robust periodic reasoning capabilities. Additionally, we propose a "Resisting Logical Oblivion" optimization strategy to maintain periodic reasoning abilities during semantic alignment. Extensive experiments demonstrate the superiority of the proposed Period-LLM over existing MLLMs in periodic tasks. The code is available at https://github.com/keke-nice/Period-LLM.

Related papers

• VLT: A Vision-Language-Time Series Multimodal Foundation Model for Industrial Intelligence [8.44621217686077]
  Industrial time series serve as the foundation for Prognostics and Health Management (PHM)<n>We propose VLT, a multimodal foundation model that jointly models time-series, frequency-spectrum visual representations, and textual knowledge.<n>VLT outperforms state-of-the-art methods, achieving superior robustness and generalization under few-shot, noisy, and incomplete-modality settings.
  arXiv  Detail & Related papers  (2026-07-16T02:53:54Z)
• UniT: Unified Multimodal Chain-of-Thought Test-time Scaling [85.590774707406]
  Unified models can handle both multimodal understanding and generation within a single architecture, yet they typically operate in a single pass without iteratively refining their outputs.<n>We introduce UniT, a framework for multimodal test-time scaling that enables a single unified model to reason, verify, and refine across multiple rounds.
  arXiv  Detail & Related papers  (2026-02-12T18:59:49Z)
• Multi-scale hypergraph meets LLMs: Aligning large language models for time series analysis [14.045113722315579]
  We propose MSH-LLM, a Multi-Scale Hypergraph method that aligns Large Language Models for time series analysis.<n> Specifically, a hyperedging mechanism is designed to enhance the multi-scale semantic information of time series semantic space.<n>A mixture of prompts (MoP) mechanism is introduced to provide contextual information and enhance the ability of LLMs to understand the multi-scale temporal patterns of time series.
  arXiv  Detail & Related papers  (2026-02-04T09:47:00Z)
• From Consistency to Complementarity: Aligned and Disentangled Multi-modal Learning for Time Series Understanding and Reasoning [12.903267405917388]
  We propose MADI, a multi-modal large language model (MLLM) enhanced with fine-grained alignment and disentangled interaction.<n>Experiments on synthetic and real-world benchmarks show that MADI consistently outperforms general-purpose LLMs and time-series-specialized MLLMs.
  arXiv  Detail & Related papers  (2026-01-29T09:13:46Z)
• TempR1: Improving Temporal Understanding of MLLMs via Temporal-Aware Multi-Task Reinforcement Learning [25.848638804759872]
  Enhancing the temporal understanding of Multimodal Large Language Models (MLLMs) is essential for advancing long-form video analysis.<n>We present TempR1, a temporal-aware multi-task reinforcement learning framework that systematically strengthens MLLMs' temporal comprehension.
  arXiv  Detail & Related papers  (2025-12-03T16:57:00Z)
• Training-free Uncertainty Guidance for Complex Visual Tasks with MLLMs [61.64185573373394]
  We propose a training-free framework that uses an MLLM's intrinsic uncertainty as a proactive guidance signal.<n>We introduce a unified mechanism that scores candidate visual inputs by response uncertainty, enabling the model to autonomously focus on the most salient data.<n>Our work validates that harnessing intrinsic uncertainty is a powerful, general strategy for enhancing fine-grained multimodal performance.
  arXiv  Detail & Related papers  (2025-10-01T09:20:51Z)
• Semantic-Enhanced Time-Series Forecasting via Large Language Models [20.383296465541758]
  Time series forecasting plays a significant role in finance, energy, meteorology, and IoT applications.<n>Recent studies have leveraged the generalization capabilities of large language models (LLMs) to adapt to time series forecasting, achieving promising performance.<n>We propose a novel Semantic-Enhanced LLM (SE-LLM) that explores the inherent periodicity and anomalous characteristics of time series to embed into the semantic space.
  arXiv  Detail & Related papers  (2025-08-11T07:19:21Z)
• Adapting LLMs to Time Series Forecasting via Temporal Heterogeneity Modeling and Semantic Alignment [32.41581846555808]
  Large Language Models (LLMs) have recently demonstrated impressive capabilities in natural language processing.<n>We propose TALON, a unified framework that enhances LLM-based forecasting by modeling temporal and enforcing semantic alignment.<n>Experiments on seven real-world benchmarks demonstrate that TALON achieves superior performance across all datasets.
  arXiv  Detail & Related papers  (2025-08-10T06:06:19Z)
• TAMMs: Temporal-Aware Multimodal Model for Satellite Image Change Understanding and Forecasting [8.914172086217185]
  We study the capabilities of multimodal large language models (MLLMs) on a novel task that jointly targets temporal change understanding and future scene generation.<n>We propose TAMMs, a Temporal-Aware Multimodal Model for satellite image understanding and forecasting.
  arXiv  Detail & Related papers  (2025-06-23T17:26:16Z)
• EIFBENCH: Extremely Complex Instruction Following Benchmark for Large Language Models [65.48902212293903]
  We present the Extremely Complex Instruction Following Benchmark (EIFBENCH) for evaluating large language models (LLMs)<n>EIFBENCH includes multi-task scenarios that enable comprehensive assessment across diverse task types concurrently.<n>We also propose the Segment Policy Optimization (SegPO) algorithm to enhance the LLM's ability to accurately fulfill multi-task workflow.
  arXiv  Detail & Related papers  (2025-06-10T02:39:55Z)
• MLLMs are Deeply Affected by Modality Bias [158.64371871084478]
  Recent advances in Multimodal Large Language Models (MLLMs) have shown promising results in integrating diverse modalities such as texts and images.<n>MLLMs are heavily influenced by modality bias, often relying on language while under-utilizing other modalities like visual inputs.<n>This paper argues that MLLMs are deeply affected by modality bias, highlighting its manifestations across various tasks.
  arXiv  Detail & Related papers  (2025-05-24T11:49:31Z)
• LLM-PS: Empowering Large Language Models for Time Series Forecasting with Temporal Patterns and Semantics [56.99021951927683]
  Time Series Forecasting (TSF) is critical in many real-world domains like financial planning and health monitoring.<n>Existing Large Language Models (LLMs) usually perform suboptimally because they neglect the inherent characteristics of time series data.<n>We propose LLM-PS to empower the LLM for TSF by learning the fundamental textitPatterns and meaningful textitSemantics from time series data.
  arXiv  Detail & Related papers  (2025-03-12T11:45:11Z)
• Position: Empowering Time Series Reasoning with Multimodal LLMs [49.73647759532127]
  We argue that multimodal language models (MLLMs) can enable more powerful and flexible reasoning for time series analysis.<n>We call on researchers and practitioners to leverage this potential by developing strategies that prioritize trust, interpretability, and robust reasoning in MLLMs.
  arXiv  Detail & Related papers  (2025-02-03T16:10:48Z)
• TempoGPT: Enhancing Time Series Reasoning via Quantizing Embedding [13.996105878417204]
  We propose a multi-modal time series data construction approach and a multi-modal time series language model (TLM), TempoGPT.<n>We construct multi-modal data for complex reasoning tasks by analyzing the variable-system relationships within a white-box system.<n>Extensive experiments demonstrate that TempoGPT accurately perceives temporal information, logically infers conclusions, and achieves state-of-the-art in the constructed complex time series reasoning tasks.
  arXiv  Detail & Related papers  (2025-01-13T13:47:05Z)
• Multimodal Large Models Are Effective Action Anticipators [10.454791411515812]
  ActionLLM is a novel approach that treats video sequences as successive tokens, leveraging Large Language Models to anticipate future actions.<n>Our baseline model simplifies the LLM architecture by setting future tokens, incorporating an action tuning module, and reducing the textual decoder layer to a linear layer.<n>To further harness the commonsense reasoning of LLMs, we predict action categories for observed frames and use sequential textual clues to guide semantic understanding.
  arXiv  Detail & Related papers  (2025-01-01T10:16:10Z)
• InternLM-XComposer2.5-OmniLive: A Comprehensive Multimodal System for Long-term Streaming Video and Audio Interactions [104.90258030688256]
  This project introduces disentangled streaming perception, reasoning, and memory mechanisms, enabling real-time interaction with streaming video and audio input.<n>This project simulates human-like cognition, enabling multimodal large language models to provide continuous and adaptive service over time.
  arXiv  Detail & Related papers  (2024-12-12T18:58:30Z)
• A Comprehensive Review of Multimodal Large Language Models: Performance and Challenges Across Different Tasks [74.52259252807191]
  Multimodal Large Language Models (MLLMs) address the complexities of real-world applications far beyond the capabilities of single-modality systems. This paper systematically sorts out the applications of MLLM in multimodal tasks such as natural language, vision, and audio.
  arXiv  Detail & Related papers  (2024-08-02T15:14:53Z)
• Model Composition for Multimodal Large Language Models [71.5729418523411]
  We propose a new paradigm through the model composition of existing MLLMs to create a new model that retains the modal understanding capabilities of each original model. Our basic implementation, NaiveMC, demonstrates the effectiveness of this paradigm by reusing modality encoders and merging LLM parameters.
  arXiv  Detail & Related papers  (2024-02-20T06:38:10Z)
• On the Performance of Multimodal Language Models [4.677125897916577]
  This study conducts a comparative analysis of different multimodal instruction tuning approaches. We reveal key insights for guiding architectural choices when incorporating multimodal capabilities into large language models.
  arXiv  Detail & Related papers  (2023-10-04T23:33:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.