A DVDrive Approach for doScenes Instructed Driving Challenge

• URL: http://arxiv.org/abs/2606.21623v1
• Date: Fri, 19 Jun 2026 17:26:58 GMT
• Title: A DVDrive Approach for doScenes Instructed Driving Challenge
• Abstract summary: This paper presents our submission to the doScenes Instructed Driving Challenge, built upon OmniDrive.<n>We adapt OmniDrive to the doScenes setting by training it on instruction-annotated nuScenes scenes.<n>To improve multi-view visual grounding, we introduce a DVPE-style divided-view perception module.
• Score: 42.41052099695495
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Instruction-conditioned trajectory prediction is an emerging problem in autonomous driving, where a model predicts the future ego trajectory not only from visual scene context and historical motion, but also from a natural-language maneuver instruction. This paper presents our submission to the doScenes Instructed Driving Challenge, built upon OmniDrive, a vision-language-action driving agent with 3D perception, reasoning, and planning capabilities. We adapt OmniDrive to the doScenes setting by training it on instruction-annotated nuScenes scenes and generating a 6-second ego trajectory represented by 12 future waypoints. To improve multi-view visual grounding, we further introduce a DVPE-style divided-view perception module into the OmniDrive perception head. Instead of attending globally to all camera features, the proposed module groups query features and image tokens into divided local view spaces and performs visibility-aware cross-attention within each view. This design reduces irrelevant cross-view interference and helps the model better align language instructions with local driving-relevant visual evidence. The code is publicly available at: https://github.com/feel12348/doscenes-omnidrive.

Related papers

• Qwen-Drive-1.0: An Initial Step towards a Vision-Language Foundation Model for Autonomous Driving [68.31544904268218]
  We present Qwen-Drive-1.0, an initial step towards a vision-language foundation model for autonomous driving.<n>Qwen-Drive-1.0 retains the architecture of the pretrained vision-language model (VLM) and integrates 3D perception, visual question answering, and motion planning within a unified framework.
  arXiv  Detail & Related papers  (2026-08-31T17:59:54Z)
• Teaching Vision-Language-Action Models What to See and Where to Look [40.70324197562915]
  Vision-Language-Action (VLA) models have emerged as a promising paradigm for end-to-end autonomous driving.<n>We propose DriveTeach-VLA, a framework that explicitly teaches VLAs what to see and where to look.
  arXiv  Detail & Related papers  (2026-07-02T03:34:32Z)
• UniDriveVLA: Unifying Understanding, Perception, and Action Planning for Autonomous Driving [48.461267171124945]
  We propose UniDriveVLA, a Unified Driving Vision-Language-Action model for autonomous driving.<n>It comprises three experts for driving understanding, scene perception, and action planning, which are coordinated through masked joint attention.<n>It achieves state-of-the-art performance in open-loop evaluation on nuScenes and closed-loop evaluation on Bench2Drive.
  arXiv  Detail & Related papers  (2026-04-02T15:48:45Z)
• Vega: Learning to Drive with Natural Language Instructions [93.49357278917223]
  We propose a unified Vision-Language-World-Action model, Vega, for instruction-based generation and planning.<n>We employ the autoregressive paradigm to process visual inputs (vision) and language instructions (language) and the diffusion paradigm to generate future predictions.<n>Our method achieves superior planning performance and exhibits strong instruction-following abilities.
  arXiv  Detail & Related papers  (2026-03-26T17:59:56Z)
• Bridging Scene Generation and Planning: Driving with World Model via Unifying Vision and Motion Representation [66.7879424097418]
  We present WorldDrive, a holistic framework that couples scene generation and real-time planning via unifying vision and motion representation.<n>A simple interaction between motion representation, visual representation, and ego status can generate high-quality, multi-modal trajectories.<n>Experiments on the NAVSIM, NAVSIM-v2, and nuScenes benchmarks demonstrate that WorldDrive achieves leading planning performance among vision-only methods.
  arXiv  Detail & Related papers  (2026-03-16T07:59:39Z)
• VISTA: Vision-Language Imitation of Situational Thinking and Attention for Human-Like Driver Focus in Dynamic Environments [0.0]
  We propose a vision-language framework that models the changing landscape of drivers' gaze through natural language.<n>Our approach integrates both low-level cues and top-down context, enabling language-based descriptions of gaze behavior.<n>Results show that our fine-tuned model outperforms general-purpose VLMs in attention shift detection and interpretability.
  arXiv  Detail & Related papers  (2025-08-07T21:01:43Z)
• InsightDrive: Insight Scene Representation for End-to-End Autonomous Driving [3.8737986316149775]
  We propose a novel end-to-end autonomous driving method called InsightDrive.<n>It organizes perception by language-guided scene representation.<n>In experiments, InsightDrive achieves state-of-the-art performance in end-to-end autonomous driving.
  arXiv  Detail & Related papers  (2025-03-17T10:52:32Z)
• HERMES: A Unified Self-Driving World Model for Simultaneous 3D Scene Understanding and Generation [59.675030933810106]
  We present a unified Driving World Model named HERMES.<n>We seamlessly integrate 3D scene understanding and future scene evolution (generation) through a unified framework in driving scenarios.<n> HERMES achieves state-of-the-art performance, reducing generation error by 32.4% and improving understanding metrics such as CIDEr by 8.0%.
  arXiv  Detail & Related papers  (2025-01-24T18:59:51Z)
• Seeing Beyond Views: Multi-View Driving Scene Video Generation with Holistic Attention [61.3281618482513]
  We present CogDriving, a novel network designed for synthesizing high-quality multi-view driving videos.<n>CogDriving leverages a Diffusion Transformer architecture with holistic-4D attention modules, enabling simultaneous associations across the dimensions.<n>CogDriving demonstrates strong performance on the nuScenes validation set, achieving an FVD score of 37.8, highlighting its ability to generate realistic driving videos.
  arXiv  Detail & Related papers  (2024-12-04T18:02:49Z)
• Estimation of Appearance and Occupancy Information in Birds Eye View from Surround Monocular Images [2.69840007334476]
  Birds-eye View (BEV) expresses the location of different traffic participants in the ego vehicle frame from a top-down view. We propose a novel representation that captures various traffic participants appearance and occupancy information from an array of monocular cameras covering 360 deg field of view (FOV) We use a learned image embedding of all camera images to generate a BEV of the scene at any instant that captures both appearance and occupancy of the scene.
  arXiv  Detail & Related papers  (2022-11-08T20:57:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.