Cut to the Chase: Training-free Multimodal Summarization via Chain-of-Events

• URL: http://arxiv.org/abs/2603.06213v1
• Date: Fri, 06 Mar 2026 12:29:33 GMT
• Title: Cut to the Chase: Training-free Multimodal Summarization via Chain-of-Events
• Abstract summary: Multimodal Summarization (MMS) aims to generate concise textual summaries by understanding and integrating information across videos, transcripts, and images.<n>Existing approaches still suffer from three main challenges: (1) reliance on domain-specific supervision, (2) implicit fusion with weak cross-modal grounding, and (3) flat temporal modeling without event transitions.<n>To address these issues, we introduce **CoE**, a training-free MMS framework that performs structured reasoning through a **Chain-of-Events** guided by a Hierarchical Event Graph (HEG).
• Score: 53.2930351741501
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Multimodal Summarization (MMS) aims to generate concise textual summaries by understanding and integrating information across videos, transcripts, and images. However, existing approaches still suffer from three main challenges: (1) reliance on domain-specific supervision, (2) implicit fusion with weak cross-modal grounding, and (3) flat temporal modeling without event transitions. To address these issues, we introduce **CoE**, a training-free MMS framework that performs structured reasoning through a **Chain-of-Events** guided by a Hierarchical Event Graph (HEG). The HEG encodes textual semantics into an explicit event hierarchy that scaffolds cross-modal grounding and temporal reasoning. Guided by this structure, **CoE** localizes key visual cues, models event evolution and causal transitions, and refines outputs via lightweight style adaptation for domain alignment. Extensive experiments on eight diverse datasets demonstrate that **CoE** consistently outperforms state-of-the-art video CoT baselines, achieving average gains of **+3.04 ROUGE**, **+9.51 CIDEr**, and **+1.88 BERTScore**, highlighting its robustness, interpretability, and cross-domain generalization. Our code is available at https://github.com/youxiaoxing/CoE.

Related papers

• Toward Federated Multimodal Graph Foundation Models: A Topology-Aware Multimodal Alignment Framework [35.35423006939282]
  FedGAMMA casts federated multimodal graph foundation learning as a two-stage semantic-structural alignment problem.<n>During pre-training, a shared-private semantic enhancer disentangles cross-modal commonality from modality-specific information.<n>During fine-tuning, FedGAMMA adapts the pretrained encoder through lightweight graph-aware prompts.
  arXiv  Detail & Related papers  (2026-07-17T06:57:41Z)
• RMPL: Relation-aware Multi-task Progressive Learning with Stage-wise Training for Multimedia Event Extraction [8.874574211900562]
  We propose a Relation-aware Multi-task Progressive Learning framework for Multimedia Event Extraction.<n>RMPL incorporates heterogeneous supervision from unimodal event extraction and multimedia relation extraction with stage-wise training.<n> Experiments on the M2E2 benchmark with multiple VLMs show consistent improvements across different modality settings.
  arXiv  Detail & Related papers  (2026-02-14T12:43:25Z)
• DynaPURLS: Dynamic Refinement of Part-aware Representations for Skeleton-based Zero-Shot Action Recognition [51.80782323686666]
  We introduce textbfDynaPURLS, a unified framework that establishes robust, multi-scale visual-semantic correspondences.<n>Our framework leverages a large language model to generate hierarchical textual descriptions that encompass both global movements and local body-part dynamics.<n>Experiments on three large-scale benchmark datasets, including NTU RGB+D 60/120 and PKU-MMD, demonstrate that DynaPURLS significantly outperforms prior art.
  arXiv  Detail & Related papers  (2025-12-12T10:39:10Z)
• Do We Need Perfect Data? Leveraging Noise for Domain Generalized Segmentation [20.89655949578527]
  This paper presents FLEX-Seg, a framework that transforms this limitation into an opportunity for robust learning.<n>Experiments across five real-world datasets demonstrate consistent improvements over state-of-the-art methods.<n>Our findings validate that adaptive strategies for handling imperfect synthetic data lead to superior domain generalization.
  arXiv  Detail & Related papers  (2025-11-28T07:46:32Z)
• The Telephone Game: Evaluating Semantic Drift in Unified Models [41.650904633974584]
  Employing a single, unified model (UM) for both visual understanding (image-to-text: I2T) and visual generation (text-to-image: T2I) has opened a new direction in Visual Language Model (VLM) research.<n>Existing evaluation benchmarks consider these capabilities in isolation: FID and GenEval for T2I, and benchmarks such as MME, MMBench for I2T.<n>These isolated single-pass metrics do not reveal cross-consistency: whether a model that "understands" a concept can also "render" it, nor whether semantic meaning
  arXiv  Detail & Related papers  (2025-09-04T17:53:52Z)
• E-CaTCH: Event-Centric Cross-Modal Attention with Temporal Consistency and Class-Imbalance Handling for Misinformation Detection [4.34871407874107]
  E-CaTCH is a framework for robustly detecting misinformation on social media.<n>It clusters posts into pseudo-events based on textual similarity and temporal proximity, then processes each event independently.<n>E-CaTCH consistently outperforms state-of-the-art baselines.
  arXiv  Detail & Related papers  (2025-08-15T04:13:23Z)
• ESTR-CoT: Towards Explainable and Accurate Event Stream based Scene Text Recognition with Chain-of-Thought Reasoning [57.767536707234036]
  We propose a novel chain-of-thought reasoning based event stream scene text recognition framework, termed ESTR-CoT.<n>Specifically, we first adopt the vision encoder EVA-CLIP to transform the input event stream into tokens and utilize a Llama tokenizer to encode the given generation prompt.<n>A Q-former is used to align the vision token to the pre-trained large language model Vicuna-7B and output both the answer and chain-of-thought (CoT) reasoning process simultaneously.
  arXiv  Detail & Related papers  (2025-07-02T23:41:31Z)
• Compile Scene Graphs with Reinforcement Learning [69.36723767339001]
  Next-token prediction is the fundamental principle for training large language models (LLMs)<n>We introduce R1-SGG, a multimodal LLM (M-LLM) initially trained via supervised fine-tuning (SFT) on the scene graph dataset.<n>We design a set of graph-centric rewards, including three recall-based variants -- Hard Recall, Hard Recall+Relax, and Soft Recall.
  arXiv  Detail & Related papers  (2025-04-18T10:46:22Z)
• Unlocking Multi-Modal Potentials for Link Prediction on Dynamic Text-Attributed Graphs [28.533930417703715]
  Dynamic Text-Attributed Graphs (DyTAGs) are a novel graph paradigm that captures evolving temporal events (edges) alongside rich textual attributes.<n>MoMent is a multi-modal model that explicitly models, integrates, and aligns each modality to learn node representations for link prediction.<n>Experiments show that MoMent achieves up to 17.28% accuracy improvement and up to 31x speed-up against eight baselines.
  arXiv  Detail & Related papers  (2025-02-27T00:49:44Z)
• NativE: Multi-modal Knowledge Graph Completion in the Wild [51.80447197290866]
  We propose a comprehensive framework NativE to achieve MMKGC in the wild. NativE proposes a relation-guided dual adaptive fusion module that enables adaptive fusion for any modalities. We construct a new benchmark called WildKGC with five datasets to evaluate our method.
  arXiv  Detail & Related papers  (2024-03-28T03:04:00Z)
• Object Segmentation by Mining Cross-Modal Semantics [68.88086621181628]
  We propose a novel approach by mining the Cross-Modal Semantics to guide the fusion and decoding of multimodal features. Specifically, we propose a novel network, termed XMSNet, consisting of (1) all-round attentive fusion (AF), (2) coarse-to-fine decoder (CFD), and (3) cross-layer self-supervision.
  arXiv  Detail & Related papers  (2023-05-17T14:30:11Z)
• Jointly Cross- and Self-Modal Graph Attention Network for Query-Based Moment Localization [77.21951145754065]
  We propose a novel Cross- and Self-Modal Graph Attention Network (CSMGAN) that recasts this task as a process of iterative messages passing over a joint graph. Our CSMGAN is able to effectively capture high-order interactions between two modalities, thus enabling a further precise localization.
  arXiv  Detail & Related papers  (2020-08-04T08:25:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.