Scene Informer: Anchor-based Occlusion Inference and Trajectory Prediction in Partially Observable Environments

• URL: http://arxiv.org/abs/2309.13893v3
• Date: Sat, 9 Mar 2024 00:40:08 GMT
• Title: Scene Informer: Anchor-based Occlusion Inference and Trajectory Prediction in Partially Observable Environments
• Authors: Bernard Lange, Jiachen Li, and Mykel J. Kochenderfer
• Abstract summary: Navigating complex and dynamic environments requires autonomous vehicles to reason about both visible and occluded regions. This involves predicting the future motion of observed agents, inferring occluded ones, and modeling their interactions. We introduce the Scene Informer, a unified approach for predicting both observed agent trajectories and inferring occludeds in a partially observable setting.
• Score: 43.28710918095309
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Navigating complex and dynamic environments requires autonomous vehicles (AVs) to reason about both visible and occluded regions. This involves predicting the future motion of observed agents, inferring occluded ones, and modeling their interactions based on vectorized scene representations of the partially observable environment. However, prior work on occlusion inference and trajectory prediction have developed in isolation, with the former based on simplified rasterized methods and the latter assuming full environment observability. We introduce the Scene Informer, a unified approach for predicting both observed agent trajectories and inferring occlusions in a partially observable setting. It uses a transformer to aggregate various input modalities and facilitate selective queries on occlusions that might intersect with the AV's planned path. The framework estimates occupancy probabilities and likely trajectories for occlusions, as well as forecast motion for observed agents. We explore common observability assumptions in both domains and their performance impact. Our approach outperforms existing methods in both occupancy prediction and trajectory prediction in partially observable setting on the Waymo Open Motion Dataset.

Related papers

• Self-supervised Multi-future Occupancy Forecasting for Autonomous Driving [45.886941596233974]
  LiDAR-generated occupancy grid maps (L-OGMs) offer a robust bird's-eye view for the scene representation. Our proposed framework performs L-OGM prediction in the latent space of a generative architecture. We decode predictions using either a single-step decoder, which provides high-quality predictions in real-time, or a diffusion-based batch decoder.
  arXiv  Detail & Related papers  (2024-07-30T18:37:59Z)
• Enhancing Trajectory Prediction through Self-Supervised Waypoint Noise Prediction [9.385936248154987]
  Trajectory prediction is an important task that involves modeling the indeterminate nature of traffic actors to forecast future trajectories. We propose a novel approach called SSWNP (Self-Supervised Waypoint Noise Prediction) In our approach, we first create clean and noise-augmented views of past observed trajectories across the spatial domain of waypoints.
  arXiv  Detail & Related papers  (2023-11-26T19:03:41Z)
• Implicit Occupancy Flow Fields for Perception and Prediction in Self-Driving [68.95178518732965]
  A self-driving vehicle (SDV) must be able to perceive its surroundings and predict the future behavior of other traffic participants. Existing works either perform object detection followed by trajectory of the detected objects, or predict dense occupancy and flow grids for the whole scene. This motivates our unified approach to perception and future prediction that implicitly represents occupancy and flow over time with a single neural network.
  arXiv  Detail & Related papers  (2023-08-02T23:39:24Z)
• Towards Motion Forecasting with Real-World Perception Inputs: Are End-to-End Approaches Competitive? [93.10694819127608]
  We propose a unified evaluation pipeline for forecasting methods with real-world perception inputs. Our in-depth study uncovers a substantial performance gap when transitioning from curated to perception-based data.
  arXiv  Detail & Related papers  (2023-06-15T17:03:14Z)
• Bridging the Gap Between Multi-Step and One-Shot Trajectory Prediction via Self-Supervision [2.365702128814616]
  Accurate vehicle trajectory prediction is an unsolved problem in autonomous driving. We propose a middle-ground where multiple trajectory segments are chained together. Our proposed Multi-Branch Self-Supervised Predictor receives additional training on new predictions starting at intermediate future segments.
  arXiv  Detail & Related papers  (2023-06-06T02:46:28Z)
• Joint Forecasting of Panoptic Segmentations with Difference Attention [72.03470153917189]
  We study a new panoptic segmentation forecasting model that jointly forecasts all object instances in a scene. We evaluate the proposed model on the Cityscapes and AIODrive datasets.
  arXiv  Detail & Related papers  (2022-04-14T17:59:32Z)
• Trajectory Forecasting from Detection with Uncertainty-Aware Motion Encoding [121.66374635092097]
  Trajectories obtained from object detection and tracking are inevitably noisy. We propose a trajectory predictor directly based on detection results without relying on explicitly formed trajectories.
  arXiv  Detail & Related papers  (2022-02-03T09:09:56Z)
• Pose and Semantic Map Based Probabilistic Forecast of Vulnerable Road Users' Trajectories [2.984037222955095]
  An approach for probabilistic trajectory forecasting of vulnerable road users (VRUs) is presented. Past movements are represented by 3D poses reflecting the posture and movements of individual body parts. The surrounding scene is modeled in the form of semantic maps showing, e.g., the course of streets, sidewalks, and the occurrence of obstacles.
  arXiv  Detail & Related papers  (2021-06-04T16:56:13Z)
• Panoptic Segmentation Forecasting [71.75275164959953]
  Our goal is to forecast the near future given a set of recent observations. We think this ability to forecast, i.e., to anticipate, is integral for the success of autonomous agents. We develop a two-component model: one component learns the dynamics of the background stuff by anticipating odometry, the other one anticipates the dynamics of detected things.
  arXiv  Detail & Related papers  (2021-04-08T17:59:16Z)
• Prediction by Anticipation: An Action-Conditional Prediction Method based on Interaction Learning [23.321627835039934]
  We propose prediction by anticipation, which views interaction in terms of a latent probabilistic generative process. Under this view, consecutive data frames can be factorized into sequential samples from an action-conditional distribution. Our proposed prediction model, variational Bayesian in nature, is trained to maximize the evidence lower bound (ELBO) of this conditional distribution.
  arXiv  Detail & Related papers  (2020-12-25T01:39:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.