GHOST: Ground-projected Hypotheses from Observed Structure-from-Motion Trajectories

• URL: http://arxiv.org/abs/2603.20583v2
• Date: Tue, 24 Mar 2026 06:38:04 GMT
• Title: GHOST: Ground-projected Hypotheses from Observed Structure-from-Motion Trajectories
• Abstract summary: We present a scalable self-supervised approach for segmenting vehicle trajectories from monocular images for autonomous driving.<n>We treat recorded ego-vehicle motion as implicit supervision and recover camera trajectories via monocular structure-from-motion.<n>Our results indicate that large-scale ego-motion distillation yields structured and generalizable path proposals.
• Score: 1.9318925338937918
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present a scalable self-supervised approach for segmenting feasible vehicle trajectories from monocular images for autonomous driving in complex urban environments. Leveraging large-scale dashcam videos, we treat recorded ego-vehicle motion as implicit supervision and recover camera trajectories via monocular structure-from-motion, projecting them onto the ground plane to generate spatial masks of traversed regions without manual annotation. These automatically generated labels are used to train a deep segmentation network that predicts motion-conditioned path proposals from a single RGB image at run time, without explicit modeling of road or lane markings. Trained on diverse, unconstrained internet data, the model implicitly captures scene layout, lane topology, and intersection structure, and generalizes across varying camera configurations. We evaluate our approach on NuScenes, demonstrating reliable trajectory prediction, and further show transfer to an electric scooter platform through light fine-tuning. Our results indicate that large-scale ego-motion distillation yields structured and generalizable path proposals beyond the demonstrated trajectory, enabling trajectory hypothesis estimation via image segmentation.

Related papers

• Infrastructure-based Monocular 3D Vehicle Localization Framework with Experimental Validation [0.0]
  This paper presents a one-stage learning framework that maps monocular roadside-camera images directly to vehicle states in a ground-fixed coordinate frame.<n>We develop a data-collection and label-generation pipeline based on synchronized video from a roadside camera and an unmanned aerial vehicle.<n>Results show that the proposed method can recover vehicle trajectories and orientations from monocular roadside imagery without a separate geometric post-processing stage.
  arXiv  Detail & Related papers  (2026-09-01T09:21:14Z)
• ReinDriveGen: Reinforcement Post-Training for Out-of-Distribution Driving Scene Generation [61.64162666162948]
  ReinDriveGen is a framework that enables full controllability over dynamic driving scenes.<n>Our approach constructs a dynamic 3D point cloud scene from multi-frame LiDAR data.<n>Our approach renders the edited scene into 2D condition images that guide a video diffusion model to synthesize realistic driving videos.
  arXiv  Detail & Related papers  (2026-04-01T16:48:20Z)
• Diffusion-FS: Multimodal Free-Space Prediction via Diffusion for Autonomous Driving [7.667821982085968]
  Drivable free-space prediction is a fundamental and crucial problem in autonomous driving.<n>Recent works have addressed the problem by representing the entire non-obstacle road regions as the free-space.<n>Our aim is to estimate the driving corridors that are a navigable subset of the entire road region.
  arXiv  Detail & Related papers  (2025-07-24T19:30:55Z)
• DrivingForward: Feed-forward 3D Gaussian Splatting for Driving Scene Reconstruction from Flexible Surround-view Input [45.04354435388718]
  We propose a feed-forward Gaussian Splatting model that reconstructs driving scenes from flexible surround-view input.<n>We jointly train a pose network, a depth network, and a Gaussian network to predict the primitives that represent the driving scenes.<n>Our model outperforms existing state-of-the-art feed-forward and scene-optimized reconstruction methods in terms of reconstruction.
  arXiv  Detail & Related papers  (2024-09-19T13:16:04Z)
• Homography Guided Temporal Fusion for Road Line and Marking Segmentation [73.47092021519245]
  Road lines and markings are frequently occluded in the presence of moving vehicles, shadow, and glare. We propose a Homography Guided Fusion (HomoFusion) module to exploit temporally-adjacent video frames for complementary cues. We show that exploiting available camera intrinsic data and ground plane assumption for cross-frame correspondence can lead to a light-weight network with significantly improved performances in speed and accuracy.
  arXiv  Detail & Related papers  (2024-04-11T10:26:40Z)
• BEVSeg2TP: Surround View Camera Bird's-Eye-View Based Joint Vehicle Segmentation and Ego Vehicle Trajectory Prediction [4.328789276903559]
  Trajectory prediction is a key task for vehicle autonomy. There is a growing interest in learning-based trajectory prediction. We show that there is the potential to improve the performance of perception.
  arXiv  Detail & Related papers  (2023-12-20T15:02:37Z)
• SEPT: Towards Efficient Scene Representation Learning for Motion Prediction [19.111948522155004]
  This paper presents SEPT, a modeling framework that leverages self-supervised learning to develop powerful models for complex traffic scenes. experiments demonstrate that SEPT, without elaborate architectural design or feature engineering, achieves state-of-the-art performance on the Argoverse 1 and Argoverse 2 motion forecasting benchmarks.
  arXiv  Detail & Related papers  (2023-09-26T21:56:03Z)
• Street-View Image Generation from a Bird's-Eye View Layout [95.36869800896335]
  Bird's-Eye View (BEV) Perception has received increasing attention in recent years. Data-driven simulation for autonomous driving has been a focal point of recent research. We propose BEVGen, a conditional generative model that synthesizes realistic and spatially consistent surrounding images.
  arXiv  Detail & Related papers  (2023-01-11T18:39:34Z)
• Monocular BEV Perception of Road Scenes via Front-to-Top View Projection [57.19891435386843]
  We present a novel framework that reconstructs a local map formed by road layout and vehicle occupancy in the bird's-eye view. Our model runs at 25 FPS on a single GPU, which is efficient and applicable for real-time panorama HD map reconstruction.
  arXiv  Detail & Related papers  (2022-11-15T13:52:41Z)
• SurroundDepth: Entangling Surrounding Views for Self-Supervised Multi-Camera Depth Estimation [101.55622133406446]
  We propose a SurroundDepth method to incorporate the information from multiple surrounding views to predict depth maps across cameras. Specifically, we employ a joint network to process all the surrounding views and propose a cross-view transformer to effectively fuse the information from multiple views. In experiments, our method achieves the state-of-the-art performance on the challenging multi-camera depth estimation datasets.
  arXiv  Detail & Related papers  (2022-04-07T17:58:47Z)
• Structured Bird's-Eye-View Traffic Scene Understanding from Onboard Images [128.881857704338]
  We study the problem of extracting a directed graph representing the local road network in BEV coordinates, from a single onboard camera image. We show that the method can be extended to detect dynamic objects on the BEV plane. We validate our approach against powerful baselines and show that our network achieves superior performance.
  arXiv  Detail & Related papers  (2021-10-05T12:40:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.