BEVSeg2TP: Surround View Camera Bird's-Eye-View Based Joint Vehicle Segmentation and Ego Vehicle Trajectory Prediction

• URL: http://arxiv.org/abs/2312.13081v1
• Date: Wed, 20 Dec 2023 15:02:37 GMT
• Title: BEVSeg2TP: Surround View Camera Bird's-Eye-View Based Joint Vehicle Segmentation and Ego Vehicle Trajectory Prediction
• Authors: Sushil Sharma, Arindam Das, Ganesh Sistu, Mark Halton, Ciar\'an Eising
• Abstract summary: Trajectory prediction is a key task for vehicle autonomy. There is a growing interest in learning-based trajectory prediction. We show that there is the potential to improve the performance of perception.
• Score: 4.328789276903559
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Trajectory prediction is, naturally, a key task for vehicle autonomy. While the number of traffic rules is limited, the combinations and uncertainties associated with each agent's behaviour in real-world scenarios are nearly impossible to encode. Consequently, there is a growing interest in learning-based trajectory prediction. The proposed method in this paper predicts trajectories by considering perception and trajectory prediction as a unified system. In considering them as unified tasks, we show that there is the potential to improve the performance of perception. To achieve these goals, we present BEVSeg2TP - a surround-view camera bird's-eye-view-based joint vehicle segmentation and ego vehicle trajectory prediction system for autonomous vehicles. The proposed system uses a network trained on multiple camera views. The images are transformed using several deep learning techniques to perform semantic segmentation of objects, including other vehicles, in the scene. The segmentation outputs are fused across the camera views to obtain a comprehensive representation of the surrounding vehicles from the bird's-eye-view perspective. The system further predicts the future trajectory of the ego vehicle using a spatiotemporal probabilistic network (STPN) to optimize trajectory prediction. This network leverages information from encoder-decoder transformers and joint vehicle segmentation.

Related papers

• PIP-Net: Pedestrian Intention Prediction in the Wild [11.799731429829603]
  PIP-Net is a novel framework designed to predict pedestrian crossing intentions by AVs in real-world urban scenarios. We offer two variants of PIP-Net designed for different camera mounts and setups. The proposed model employs a recurrent and temporal attention-based solution, outperforming state-of-the-art performance. For the first time, we present the Urban-PIP dataset, a customised pedestrian intention prediction dataset.
  arXiv  Detail & Related papers  (2024-02-20T08:28:45Z)
• Implicit Occupancy Flow Fields for Perception and Prediction in Self-Driving [68.95178518732965]
  A self-driving vehicle (SDV) must be able to perceive its surroundings and predict the future behavior of other traffic participants. Existing works either perform object detection followed by trajectory of the detected objects, or predict dense occupancy and flow grids for the whole scene. This motivates our unified approach to perception and future prediction that implicitly represents occupancy and flow over time with a single neural network.
  arXiv  Detail & Related papers  (2023-08-02T23:39:24Z)
• Street-View Image Generation from a Bird's-Eye View Layout [95.36869800896335]
  Bird's-Eye View (BEV) Perception has received increasing attention in recent years. Data-driven simulation for autonomous driving has been a focal point of recent research. We propose BEVGen, a conditional generative model that synthesizes realistic and spatially consistent surrounding images.
  arXiv  Detail & Related papers  (2023-01-11T18:39:34Z)
• Multi-Vehicle Trajectory Prediction at Intersections using State and Intention Information [50.40632021583213]
  Traditional approaches to prediction of future trajectory of road agents rely on knowing information about their past trajectory. This work instead relies on having knowledge of the current state and intended direction to make predictions for multiple vehicles at intersections. Message passing of this information between the vehicles provides each one of them a more holistic overview of the environment.
  arXiv  Detail & Related papers  (2023-01-06T15:13:23Z)
• Policy Pre-training for End-to-end Autonomous Driving via Self-supervised Geometric Modeling [96.31941517446859]
  We propose PPGeo (Policy Pre-training via Geometric modeling), an intuitive and straightforward fully self-supervised framework curated for the policy pretraining in visuomotor driving. We aim at learning policy representations as a powerful abstraction by modeling 3D geometric scenes on large-scale unlabeled and uncalibrated YouTube driving videos. In the first stage, the geometric modeling framework generates pose and depth predictions simultaneously, with two consecutive frames as input. In the second stage, the visual encoder learns driving policy representation by predicting the future ego-motion and optimizing with the photometric error based on current visual observation only.
  arXiv  Detail & Related papers  (2023-01-03T08:52:49Z)
• Monocular BEV Perception of Road Scenes via Front-to-Top View Projection [57.19891435386843]
  We present a novel framework that reconstructs a local map formed by road layout and vehicle occupancy in the bird's-eye view. Our model runs at 25 FPS on a single GPU, which is efficient and applicable for real-time panorama HD map reconstruction.
  arXiv  Detail & Related papers  (2022-11-15T13:52:41Z)
• RSG-Net: Towards Rich Sematic Relationship Prediction for Intelligent Vehicle in Complex Environments [72.04891523115535]
  We propose RSG-Net (Road Scene Graph Net): a graph convolutional network designed to predict potential semantic relationships from object proposals. The experimental results indicate that this network, trained on Road Scene Graph dataset, could efficiently predict potential semantic relationships among objects around the ego-vehicle.
  arXiv  Detail & Related papers  (2022-07-16T12:40:17Z)
• Vehicle Trajectory Prediction on Highways Using Bird Eye View Representations and Deep Learning [0.5420492913071214]
  This work presents a novel method for predicting vehicle trajectories in highway scenarios using efficient bird's eye view representations and convolutional neural networks. The U-net model has been selected as the prediction kernel to generate future visual representations of the scene using an image-to-image regression approach. A method has been implemented to extract vehicle positions from the generated graphical representations to achieve subpixel resolution.
  arXiv  Detail & Related papers  (2022-07-04T13:39:46Z)
• Vehicle Trajectory Prediction in Crowded Highway Scenarios Using Bird Eye View Representations and CNNs [0.0]
  This paper describes a novel approach to perform vehicle trajectory predictions employing graphic representations. The problem is faced as an image to image regression problem training the network to learn the underlying relations between the traffic participants. The model has been tested in highway scenarios with more than 30 vehicles simultaneously in two opposite traffic flow streams.
  arXiv  Detail & Related papers  (2020-08-26T11:15:49Z)
• Two-Stream Networks for Lane-Change Prediction of Surrounding Vehicles [8.828423067460644]
  In highway scenarios, an alert human driver will typically anticipate early cut-in and cut-out maneuvers surrounding vehicles using only visual cues. To deal with lane-change recognition and prediction of surrounding vehicles, we pose the problem as an action recognition/prediction problem by stacking visual cues from video cameras. Two video action recognition approaches are analyzed: two-stream convolutional networks and multiplier networks.
  arXiv  Detail & Related papers  (2020-08-25T07:59:15Z)
• PLOP: Probabilistic poLynomial Objects trajectory Planning for autonomous driving [8.105493956485583]
  We use a conditional imitation learning algorithm to predict trajectories for ego vehicle and its neighbors. Our approach is computationally efficient and relies only on on-board sensors. We evaluate our method offline on the publicly available dataset nuScenes.
  arXiv  Detail & Related papers  (2020-03-09T16:55:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.