MECD+: Unlocking Event-Level Causal Graph Discovery for Video Reasoning

• URL: http://arxiv.org/abs/2501.07227v3
• Date: Fri, 17 Jan 2025 02:27:29 GMT
• Title: MECD+: Unlocking Event-Level Causal Graph Discovery for Video Reasoning
• Authors: Tieyuan Chen, Huabin Liu, Yi Wang, Yihang Chen, Tianyao He, Chaofan Gan, Huanyu He, Weiyao Lin,
• Abstract summary: We introduce a new task and dataset, Multi-Event Causal Discovery (MECD) It aims to uncover the causal relations between events distributed chronologically across long videos. We devise a novel framework inspired by the Granger Causality method, incorporating an efficient mask-based event prediction model.
• Score: 16.209265930309854
• License:
• Abstract: Video causal reasoning aims to achieve a high-level understanding of videos from a causal perspective. However, it exhibits limitations in its scope, primarily executed in a question-answering paradigm and focusing on brief video segments containing isolated events and basic causal relations, lacking comprehensive and structured causality analysis for videos with multiple interconnected events. To fill this gap, we introduce a new task and dataset, Multi-Event Causal Discovery (MECD). It aims to uncover the causal relations between events distributed chronologically across long videos. Given visual segments and textual descriptions of events, MECD identifies the causal associations between these events to derive a comprehensive and structured event-level video causal graph explaining why and how the result event occurred. To address the challenges of MECD, we devise a novel framework inspired by the Granger Causality method, incorporating an efficient mask-based event prediction model to perform an Event Granger Test. It estimates causality by comparing the predicted result event when premise events are masked versus unmasked. Furthermore, we integrate causal inference techniques such as front-door adjustment and counterfactual inference to mitigate challenges in MECD like causality confounding and illusory causality. Additionally, context chain reasoning is introduced to conduct more robust and generalized reasoning. Experiments validate the effectiveness of our framework in reasoning complete causal relations, outperforming GPT-4o and VideoChat2 by 5.77% and 2.70%, respectively. Further experiments demonstrate that causal relation graphs can also contribute to downstream video understanding tasks such as video question answering and video event prediction.

Related papers

• Finding the Trigger: Causal Abductive Reasoning on Video Events [59.188208873301015]
  Causal Abductive Reasoning on Video Events (CARVE) involves identifying causal relationships between events in a video. We present a Causal Event Relation Network (CERN) that examines the relationships between video events in temporal and semantic spaces.
  arXiv  Detail & Related papers  (2025-01-16T05:39:28Z)
• MECD: Unlocking Multi-Event Causal Discovery in Video Reasoning [23.928977574352796]
  We introduce a new task and dataset, Multi-Event Causal Discovery (MECD) It aims to uncover the causal relationships between events distributed chronologically across long videos. We devise a novel framework inspired by the Granger Causality method, using an efficient mask-based event prediction model.
  arXiv  Detail & Related papers  (2024-09-26T08:51:29Z)
• Generating Event-oriented Attribution for Movies via Two-Stage Prefix-Enhanced Multimodal LLM [47.786978666537436]
  We propose a Two-Stage Prefix-Enhanced MLLM (TSPE) approach for event attribution in movie videos. In the local stage, we introduce an interaction-aware prefix that guides the model to focus on the relevant multimodal information within a single clip. In the global stage, we strengthen the connections between associated events using an inferential knowledge graph.
  arXiv  Detail & Related papers  (2024-09-14T08:30:59Z)
• Enhancing Event Causality Identification with Rationale and Structure-Aware Causal Question Answering [30.000134835133522]
  Event Causality Identification (DECI) aims to identify causal relations between two events in documents. Recent research tends to use pre-trained language models to generate the event causal relations. We propose a multi-task learning framework to enhance event causality identification with rationale and structure-aware causal question answering.
  arXiv  Detail & Related papers  (2024-03-17T07:41:58Z)
• CRAB: Assessing the Strength of Causal Relationships Between Real-world Events [20.74723427835013]
  We present CRAB, a new Causal Reasoning Assessment Benchmark designed to evaluate causal understanding of events in real-world narratives. We measure the performance of several large language models, demonstrating that most systems achieve poor performance on the task. Motivated by classical causal principles, we analyze the causal structures of groups of events in CRAB, and find that models perform worse on causal reasoning when events are derived from complex causal structures.
  arXiv  Detail & Related papers  (2023-11-07T19:00:44Z)
• Visual Causal Scene Refinement for Video Question Answering [117.08431221482638]
  We present a causal analysis of VideoQA and propose a framework for cross-modal causal reasoning, named Visual Causal Scene Refinement (VCSR) Our VCSR involves two essential modules, which refines consecutive video frames guided by the question semantics to obtain more representative segment features for causal front-door intervention. Experiments on the NExT-QA, Causal-VidQA, and MSRVTT-QA datasets demonstrate the superiority of our VCSR in discovering visual causal scene and achieving robust video question answering.
  arXiv  Detail & Related papers  (2023-05-07T09:05:19Z)
• Causalainer: Causal Explainer for Automatic Video Summarization [77.36225634727221]
  In many application scenarios, improper video summarization can have a large impact. Modeling explainability is a key concern. A Causal Explainer, dubbed Causalainer, is proposed to address this issue.
  arXiv  Detail & Related papers  (2023-04-30T11:42:06Z)
• Event Causality Extraction with Event Argument Correlations [13.403222002600558]
  Event Causality Extraction aims to extract cause-effect event causality pairs from plain texts. We propose a method with a dual grid tagging scheme to capture the intra- and inter-event argument correlations for ECE.
  arXiv  Detail & Related papers  (2023-01-27T09:48:31Z)
• EA$^2$E: Improving Consistency with Event Awareness for Document-Level Argument Extraction [52.43978926985928]
  We introduce the Event-Aware Argument Extraction (EA$2$E) model with augmented context for training and inference. Experiment results on WIKIEVENTS and ACE2005 datasets demonstrate the effectiveness of EA$2$E.
  arXiv  Detail & Related papers  (2022-05-30T04:33:51Z)
• Everything Has a Cause: Leveraging Causal Inference in Legal Text Analysis [62.44432226563088]
  Causal inference is the process of capturing cause-effect relationship among variables. We propose a novel Graph-based Causal Inference framework, which builds causal graphs from fact descriptions without much human involvement. We observe that the causal knowledge contained in GCI can be effectively injected into powerful neural networks for better performance and interpretability.
  arXiv  Detail & Related papers  (2021-04-19T16:13:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.