Driving Video Retrieval for Complex Queries with Structured Grounding

• URL: http://arxiv.org/abs/2606.09109v1
• Date: Mon, 08 Jun 2026 07:00:33 GMT
• Title: Driving Video Retrieval for Complex Queries with Structured Grounding
• Abstract summary: STRIVE-D is a data-calibrated retrieval framework for driving videos.<n>It uses weakly labeled in-domain videos to estimate when a query rule is reliable.<n>It delivers up to 84% relative improvement in top-1 accuracy over state-of-the-art methods.
• Score: 6.16173133327134
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Video retrieval at scale is central to data curation and safety validation in autonomous driving, where users want to find not only scenes but also dynamic events such as cut-ins and hard braking. Existing vision-language and keyword-based retrieval methods often miss these events because the relevant motion may not be explicitly described in text or captured by lexical overlap. Rule-based retrieval can encode such events more directly, but it is brittle: generated or hand-written rules often fail when their assumptions do not match real driving data. We propose STRIVE-D, a data-calibrated retrieval framework for driving videos. It uses weakly labeled in-domain videos to estimate when a query rule is reliable, adapt rules that mismatch observed data, and fuse calibrated rule scores with vision-language and keyword-based retrieval signals. Across three driving benchmarks, including newly released human-annotated event data on DrivingDojo, STRIVE-D delivers up to 84% relative improvement in top-1 accuracy over state-of-the-art methods.

Related papers

• TraVEL: Trajectory-Guided Video Embedding Learning for Driving-Video Retrieval [15.588457512574832]
  Efficiently retrieving relevant clips from driving logs is essential for data curation, model development, and safety analysis.<n>General-purpose embedding models often rely on shortcuts from static scene context.<n>We introduce TraVEL, a motion-aware fine-tuning framework that uses ego-trajectory similarity as a reward.
  arXiv  Detail & Related papers  (2026-08-13T17:24:23Z)
• Reasoning Text-to-Video Retrieval for Operating Room Clips via Action-Driven Digital Twins [10.60845088912767]
  We propose OR3, a text-to-video retrieval method that converts clips into action-driven digital twins (ActDTs)<n>We construct a benchmark from MM-OR with 276 implicit queries across four reasoning categories over 386 clips from robotic knee procedures.<n> OR3 achieves 57.6 R@1 and 77.3 R@5, outperforming the strongest baseline.
  arXiv  Detail & Related papers  (2026-06-15T21:06:53Z)
• R^3: Composed Video Retrieval via Reasoning-Guided Recalling and Re-ranking [73.0537447183962]
  We present a zero-shot composed video retrieval pipeline built around Reasoning-guided Recalling and Reranking.<n>The model generates a reasoning trace that describes the expected target video after applying the edit.<n>Experiments have demonstrated the effectiveness of our method in addressing this challenge.
  arXiv  Detail & Related papers  (2026-05-31T09:20:53Z)
• Revisiting Uncertainty: On Evidential Learning for Partially Relevant Video Retrieval [70.47320120707029]
  Partially relevant video retrieval aims to retrieve untrimmed videos using text queries that describe only partial content.<n>In this setting, vague queries often induce semantic ambiguity across videos, a challenge that is further exacerbated by the sparse temporal supervision within videos.<n>We propose Holmes, a hierarchical evidential learning framework that aggregates multi-granular cross-modal evidence to quantify and model uncertainty explicitly.
  arXiv  Detail & Related papers  (2026-05-07T12:06:13Z)
• Follow the Saliency: Supervised Saliency for Retrieval-augmented Dense Video Captioning [7.051314153203061]
  Existing retrieval-augmented approaches for Dense Video Captioning (DVC) often fail to achieve accurate temporal segmentation aligned with true event boundaries.<n>The proposed framework, bfSTaRC, overcomes this limitation by supervising frame-level saliency through a highlight detection module.<n>We conduct comprehensive evaluations on the YouCook2 and ViTT benchmarks, where STaRC achieves state-of-the-art performance across most of the metrics.
  arXiv  Detail & Related papers  (2026-03-12T02:37:21Z)
• Uncertainty-quantified Rollout Policy Adaptation for Unlabelled Cross-domain Temporal Grounding [59.09971455857609]
  Video Temporal Grounding aims to temporally locate video segments matching a natural language description in a long video.<n>We introduce a Data-Efficient Unlabelled Cross-domain Temporal Grounding method.<n>This method eliminates the need for target annotation and keeps both computational and storage overhead low enough to run in real time.
  arXiv  Detail & Related papers  (2025-08-08T13:47:00Z)
• Addressing Out-of-Label Hazard Detection in Dashcam Videos: Insights from the COOOL Challenge [0.0]
  This paper presents a novel approach for hazard analysis in dashcam footage.<n>It addresses the detection of driver reactions to hazards, the identification of hazardous objects, and the generation of descriptive captions.<n>Our method achieved the highest scores in the Challenge on Out-of-Label in Autonomous Driving.
  arXiv  Detail & Related papers  (2025-01-27T13:32:01Z)
• Towards Generalisable Video Moment Retrieval: Visual-Dynamic Injection to Image-Text Pre-Training [70.83385449872495]
  The correlation between the vision and text is essential for video moment retrieval (VMR) Existing methods rely on separate pre-training feature extractors for visual and textual understanding. We propose a generic method, referred to as Visual-Dynamic Injection (VDI), to empower the model's understanding of video moments.
  arXiv  Detail & Related papers  (2023-02-28T19:29:05Z)
• Learning a Grammar Inducer from Massive Uncurated Instructional Videos [118.7279072358029]
  Video-aided grammar induction aims to leverage video information for finding more accurate syntactic grammars for accompanying text. We build a new model that can better learn video-span correlation without manually designed features. Our model yields higher F1 scores than the previous state-of-the-art systems trained on in-domain data.
  arXiv  Detail & Related papers  (2022-10-22T00:22:55Z)
• A Background-Agnostic Framework with Adversarial Training for Abnormal Event Detection in Video [120.18562044084678]
  Abnormal event detection in video is a complex computer vision problem that has attracted significant attention in recent years. We propose a background-agnostic framework that learns from training videos containing only normal events.
  arXiv  Detail & Related papers  (2020-08-27T18:39:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.