Towards Effective Long-Video Event Prediction via Multi-Level Event Semantics Mining

• URL: http://arxiv.org/abs/2605.31069v1
• Date: Fri, 29 May 2026 09:38:43 GMT
• Title: Towards Effective Long-Video Event Prediction via Multi-Level Event Semantics Mining
• Abstract summary: We propose VISTA, a multi-level event semantics mining framework for long-video event prediction.<n>Initially, VISTA applies a character-centric visual prompt to precisely extract event-related visual details, enhancing detail-level semantics.<n>It then employs a knowledge-enhanced iterative retrieval strategy, guiding the LLM to progressively construct logically coherent event chains.<n>Ultimately, VISTA adopts a human-like propose-then-retrieve strategy to generate diverse future-oriented proposals and integrate multi-level clues, producing robust and accurate predictions.
• Score: 10.37877709276133
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Accurately predicting future events is fundamental to content understanding and decision-making across various domains. While prior research has primarily focused on text or short-video scenarios, long-video event prediction, characterized by vast multimodal context and more complex narratives, remains underexplored. Meanwhile, although recent Long-Video Language Models (LVLMs), built on Large Language Models (LLMs) and Vision-Language Models (VLMs), have shown promise in long-video question answering and summarization, they struggle to generalize to event prediction, as they can neither precisely extract event-related details nor perform fine-grained analysis of event development. To address this gap, we propose VISTA, a multi-level event semantics mining framework for long-video event prediction. Initially, VISTA applies a character-centric visual prompt to precisely extract event-related visual details, enhancing detail-level semantics; subsequently, it employs a knowledge-enhanced iterative retrieval strategy, guiding the LLM to progressively construct logically coherent event chains, thereby improving event-level narratives; ultimately, VISTA adopts a human-like propose-then-retrieve strategy to generate diverse future-oriented proposals and integrate multi-level clues, producing robust and accurate predictions. Extensive experiments on real-world datasets validate the effectiveness of VISTA for long-video event prediction.

Related papers

• Video-CoE: Reinforcing Video Event Prediction via Chain of Events [20.58429395226725]
  VEP requires the model to perform fine-grained temporal modeling of videos and establish logical relationships between videos and future events.<n>textbfChain textbfof textbfEvents (textbfCoE) paradigm, which constructs temporal event chains to implicitly enforce MLLM.<n>Codes and models will be released soon.
  arXiv  Detail & Related papers  (2026-03-16T07:38:15Z)
• LET-US: Long Event-Text Understanding of Scenes [23.376693904132786]
  Event cameras output event streams as sparse, asynchronous data with microsecond-level temporal resolution.<n>We introduce LET-US, a framework for long event-stream--text comprehension.<n>We use an adaptive compression mechanism to reduce the volume of input events while preserving critical visual details.
  arXiv  Detail & Related papers  (2025-08-10T16:02:41Z)
• Vision and Intention Boost Large Language Model in Long-Term Action Anticipation [39.66216219048517]
  Long-term action anticipation aims to predict future actions over an extended period.<n>Recent researches leverage large language models (LLMs) by utilizing text-based inputs which suffer severe information loss.<n>We propose a novel Intention-Conditioned Vision-Language (ICVL) model in this study that fully leverages the rich semantic information of visual data.
  arXiv  Detail & Related papers  (2025-05-03T06:33:54Z)
• EventVL: Understand Event Streams via Multimodal Large Language Model [29.23525787969373]
  We propose EventVL, the first generative event-based MLLM framework for explicit semantic understanding.<n> Specifically, to bridge the data gap for connecting different modalities semantics, we first annotate a large event-image/video-text dataset.<n>To further promote a compact semantic space, Dynamic Semantic Alignment is introduced to improve and complete sparse semantic spaces of events.
  arXiv  Detail & Related papers  (2025-01-23T14:37:21Z)
• EventGPT: Event Stream Understanding with Multimodal Large Language Models [59.65010502000344]
  Event cameras record visual information as asynchronous pixel change streams, excelling at scene perception under unsatisfactory lighting or high-dynamic conditions.<n>Existing multimodal large language models (MLLMs) concentrate on natural RGB images, failing in scenarios where event data fits better.<n>We introduce EventGPT, the first MLLM for event stream understanding.
  arXiv  Detail & Related papers  (2024-12-01T14:38:40Z)
• Grounding Partially-Defined Events in Multimodal Data [61.0063273919745]
  We introduce a multimodal formulation for partially-defined events and cast the extraction of these events as a three-stage span retrieval task. We propose a benchmark for this task, MultiVENT-G, that consists of 14.5 hours of densely annotated current event videos and 1,168 text documents, containing 22.8K labeled event-centric entities. Results illustrate the challenges that abstract event understanding poses and demonstrates promise in event-centric video-language systems.
  arXiv  Detail & Related papers  (2024-10-07T17:59:48Z)
• OpenEP: Open-Ended Future Event Prediction [57.63525290892786]
  We introduce OpenEP (an Open-Ended Future Event Prediction task), which generates flexible and diverse predictions aligned with real-world scenarios. For question construction, we pose questions from seven perspectives, including location, time, event development, event outcome, event impact, event response, and other. For outcome construction, we collect free-form text containing the outcomes as ground truth to provide semantically complete and detail-enriched outcomes.
  arXiv  Detail & Related papers  (2024-08-13T02:35:54Z)
• EA-VTR: Event-Aware Video-Text Retrieval [97.30850809266725]
  Event-Aware Video-Text Retrieval model achieves powerful video-text retrieval ability through superior video event awareness. EA-VTR can efficiently encode frame-level and video-level visual representations simultaneously, enabling detailed event content and complex event temporal cross-modal alignment.
  arXiv  Detail & Related papers  (2024-07-10T09:09:58Z)
• PALM: Predicting Actions through Language Models [74.10147822693791]
  We introduce PALM, an approach that tackles the task of long-term action anticipation. Our method incorporates an action recognition model to track previous action sequences and a vision-language model to articulate relevant environmental details. Our experimental results demonstrate that PALM surpasses the state-of-the-art methods in the task of long-term action anticipation.
  arXiv  Detail & Related papers  (2023-11-29T02:17:27Z)
• CLIP-Event: Connecting Text and Images with Event Structures [123.31452120399827]
  We propose a contrastive learning framework to enforce vision-language pretraining models. We take advantage of text information extraction technologies to obtain event structural knowledge. Experiments show that our zero-shot CLIP-Event outperforms the state-of-the-art supervised model in argument extraction.
  arXiv  Detail & Related papers  (2022-01-13T17:03:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.