CoVR-R:Reason-Aware Composed Video Retrieval

• URL: http://arxiv.org/abs/2603.20190v1
• Date: Fri, 20 Mar 2026 17:59:25 GMT
• Title: CoVR-R:Reason-Aware Composed Video Retrieval
• Abstract summary: Composed Video Retrieval (CoVR) aims to find a target video given a reference video and a textual modification.<n>Prior work assumes the modification text fully specifies the visual changes, overlooking after-effects and implicit consequences.<n>We introduce a reasoning-first, zero-shot approach that leverages large multimodal models to infer causal and temporal consequences implied by the edit.
• Score: 25.765681895305736
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Composed Video Retrieval (CoVR) aims to find a target video given a reference video and a textual modification. Prior work assumes the modification text fully specifies the visual changes, overlooking after-effects and implicit consequences (e.g., motion, state transitions, viewpoint or duration cues) that emerge from the edit. We argue that successful CoVR requires reasoning about these after-effects. We introduce a reasoning-first, zero-shot approach that leverages large multimodal models to (i) infer causal and temporal consequences implied by the edit, and (ii) align the resulting reasoned queries to candidate videos without task-specific finetuning. To evaluate reasoning in CoVR, we also propose CoVR-Reason, a benchmark that pairs each (reference, edit, target) triplet with structured internal reasoning traces and challenging distractors that require predicting after-effects rather than keyword matching. Experiments show that our zero-shot method outperforms strong retrieval baselines on recall at K and particularly excels on implicit-effect subsets. Our automatic and human analysis confirm higher step consistency and effect factuality in our retrieved results. Our findings show that incorporating reasoning into general-purpose multimodal models enables effective CoVR by explicitly accounting for causal and temporal after-effects. This reduces dependence on task-specific supervision, improves generalization to challenging implicit-effect cases, and enhances interpretability of retrieval outcomes. These results point toward a scalable and principled framework for explainable video search. The model, code, and benchmark are available at https://github.com/mbzuai-oryx/CoVR-R.

Related papers

• R^3: Composed Video Retrieval via Reasoning-Guided Recalling and Re-ranking [73.0537447183962]
  We present a zero-shot composed video retrieval pipeline built around Reasoning-guided Recalling and Reranking.<n>The model generates a reasoning trace that describes the expected target video after applying the edit.<n>Experiments have demonstrated the effectiveness of our method in addressing this challenge.
  arXiv  Detail & Related papers  (2026-05-31T09:20:53Z)
• ReTrack: Evidence-Driven Dual-Stream Directional Anchor Calibration Network for Composed Video Retrieval [24.278296673415827]
  Composed Video Retrieval (CVR) has emerged as a novel paradigm in video retrieval.<n>Traditional composition methods tend to bias the composed feature toward the reference video.<n>ReTrack is the first CVR framework that improves multi-modal query understanding by calibrating directional bias in composed features.
  arXiv  Detail & Related papers  (2026-04-20T07:17:59Z)
• VIPER: Process-aware Evaluation for Generative Video Reasoning [64.86465792516658]
  We introduce VIPER, a comprehensive benchmark spanning 16 tasks across temporal, structural, symbolic, spatial, physics, and planning reasoning.<n>Our experiments reveal that state-of-the-art video models achieve only about 20% POC@1.0 and exhibit a significant outcome-hacking.
  arXiv  Detail & Related papers  (2025-12-31T16:31:59Z)
• ReVSeg: Incentivizing the Reasoning Chain for Video Segmentation with Reinforcement Learning [44.49803237328707]
  ReVSeg executes reasoning as sequential decisions in the native interface of pretrained vision language models.<n>We employ reinforcement learning to optimize the multi-step reasoning chain, enabling the model to self-refine its decision quality from outcome-driven signals.
  arXiv  Detail & Related papers  (2025-12-02T14:44:12Z)
• CDE: Curiosity-Driven Exploration for Efficient Reinforcement Learning in Large Language Models [85.315711639214]
  We introduce Curiosity-Driven Exploration (CDE), a framework that leverages the model's own intrinsic sense of curiosity to guide exploration.<n>For the actor, we use perplexity over its generated response, and for the critic, we use the variance of value estimates from a multi-head architecture.<n>Our theoretical analysis shows that the actor-wise bonus inherently penalizes overconfident errors and promotes diversity among correct responses.
  arXiv  Detail & Related papers  (2025-09-11T17:59:17Z)
• Enhancing Partially Relevant Video Retrieval with Robust Alignment Learning [29.427720184101474]
  Partially Relevant Video Retrieval aims to retrieve videos partially relevant to a given query.<n>The core challenge lies in learning robust query-video alignment against spurious semantic correlations.<n>We propose Robust Alignment Learning framework, which explicitly models the uncertainty in data.
  arXiv  Detail & Related papers  (2025-09-01T11:30:43Z)
• Reinforcing Video Reasoning Segmentation to Think Before It Segments [67.5703457389657]
  We introduce Veason-R1, a specialized LVLM for video reasoning segmentation.<n>Veason-R1 is trained through Group Relative Policy Optimization (O) augmented with Chain-of-Thought trajectories.<n>We incorporate a holistic reward mechanism that enhances spatial alignment and temporal consistency.<n>Veason-R1 achieves state-of-the-art performance on multiple benchmarks, surpassing prior art by significant margins.
  arXiv  Detail & Related papers  (2025-08-15T15:34:56Z)
• From Play to Replay: Composed Video Retrieval for Temporally Fine-Grained Videos [48.666667545084835]
  Composed Video Retrieval (CoVR) retrieves a target video given a query video and a modification text describing the intended change.<n>We introduce TF-CoVR, the first large-scale benchmark dedicated to temporally fine-grained CoVR.<n> TF-CoVR focuses on gymnastics and diving and provides 180K triplets drawn from FineGym and FineDiving.
  arXiv  Detail & Related papers  (2025-06-05T17:31:17Z)
• VCRBench: Exploring Long-form Causal Reasoning Capabilities of Large Video Language Models [29.706347050700867]
  We introduce a novel benchmark named Video-based long-form Causal Reasoning (VCRBench)<n>VCRBench tests whether Large Video Language Models (LVLMs) can identify, reason about, and correctly sequence the events needed to accomplish a specific goal.<n>We propose Recognition-Reasoning Decomposition (RRD), a modular approach that breaks video-based causal reasoning into two sub-tasks of video recognition and causal reasoning.
  arXiv  Detail & Related papers  (2025-05-13T11:35:58Z)
• Causalainer: Causal Explainer for Automatic Video Summarization [77.36225634727221]
  In many application scenarios, improper video summarization can have a large impact. Modeling explainability is a key concern. A Causal Explainer, dubbed Causalainer, is proposed to address this issue.
  arXiv  Detail & Related papers  (2023-04-30T11:42:06Z)
• Deconfounded Video Moment Retrieval with Causal Intervention [80.90604360072831]
  We tackle the task of video moment retrieval (VMR), which aims to localize a specific moment in a video according to a textual query. Existing methods primarily model the matching relationship between query and moment by complex cross-modal interactions. We propose a causality-inspired VMR framework that builds structural causal model to capture the true effect of query and video content on the prediction.
  arXiv  Detail & Related papers  (2021-06-03T01:33:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.