Modeling 3D Pedestrian-Vehicle Interactions for Vehicle-Conditioned Pose Forecasting

• URL: http://arxiv.org/abs/2602.08962v1
• Date: Mon, 09 Feb 2026 17:58:53 GMT
• Title: Modeling 3D Pedestrian-Vehicle Interactions for Vehicle-Conditioned Pose Forecasting
• Authors: Guangxun Zhu, Xuan Liu, Nicolas Pugeault, Chongfeng Wei, Edmond S. L. Ho,
• Abstract summary: Accurately predicting pedestrian motion is crucial for safe and reliable autonomous driving in complex urban environments.<n>We present a 3D vehicle-conditioned pedestrian pose forecasting framework that explicitly incorporates surrounding vehicle information.
• Score: 13.375894985533838
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Accurately predicting pedestrian motion is crucial for safe and reliable autonomous driving in complex urban environments. In this work, we present a 3D vehicle-conditioned pedestrian pose forecasting framework that explicitly incorporates surrounding vehicle information. To support this, we enhance the Waymo-3DSkelMo dataset with aligned 3D vehicle bounding boxes, enabling realistic modeling of multi-agent pedestrian-vehicle interactions. We introduce a sampling scheme to categorize scenes by pedestrian and vehicle count, facilitating training across varying interaction complexities. Our proposed network adapts the TBIFormer architecture with a dedicated vehicle encoder and pedestrian-vehicle interaction cross-attention module to fuse pedestrian and vehicle features, allowing predictions to be conditioned on both historical pedestrian motion and surrounding vehicles. Extensive experiments demonstrate substantial improvements in forecasting accuracy and validate different approaches for modeling pedestrian-vehicle interactions, highlighting the importance of vehicle-aware 3D pose prediction for autonomous driving. Code is available at: https://github.com/GuangxunZhu/VehCondPose3D

Related papers

• GeoDrive: 3D Geometry-Informed Driving World Model with Precise Action Control [50.67481583744243]
  We introduce GeoDrive, which explicitly integrates robust 3D geometry conditions into driving world models.<n>We propose a dynamic editing module during training to enhance the renderings by editing the positions of the vehicles.<n>Our method significantly outperforms existing models in both action accuracy and 3D spatial awareness.
  arXiv  Detail & Related papers  (2025-05-28T14:46:51Z)
• Inverse++: Vision-Centric 3D Semantic Occupancy Prediction Assisted with 3D Object Detection [11.33083039877258]
  3D semantic occupancy prediction aims to forecast detailed geometric and semantic information of the surrounding environment for autonomous vehicles.<n>We introduce an additional 3D supervision signal by incorporating an additional 3D object detection auxiliary branch.<n>Our approach attains state-of-the-art results, achieving an IoU score of 31.73% and a mIoU score of 20.91%.
  arXiv  Detail & Related papers  (2025-04-07T05:08:22Z)
• Trajeglish: Traffic Modeling as Next-Token Prediction [67.28197954427638]
  A longstanding challenge for self-driving development is simulating dynamic driving scenarios seeded from recorded driving logs. We apply tools from discrete sequence modeling to model how vehicles, pedestrians and cyclists interact in driving scenarios. Our model tops the Sim Agents Benchmark, surpassing prior work along the realism meta metric by 3.3% and along the interaction metric by 9.9%.
  arXiv  Detail & Related papers  (2023-12-07T18:53:27Z)
• Pedestrian Environment Model for Automated Driving [54.16257759472116]
  We propose an environment model that includes the position of the pedestrians as well as their pose information. We extract the skeletal information with a neural network human pose estimator from the image. To obtain the 3D information of the position, we aggregate the data from consecutive frames in conjunction with the vehicle position.
  arXiv  Detail & Related papers  (2023-08-17T16:10:58Z)
• HUM3DIL: Semi-supervised Multi-modal 3D Human Pose Estimation for Autonomous Driving [95.42203932627102]
  3D human pose estimation is an emerging technology, which can enable the autonomous vehicle to perceive and understand the subtle and complex behaviors of pedestrians. Our method efficiently makes use of these complementary signals, in a semi-supervised fashion and outperforms existing methods with a large margin. Specifically, we embed LiDAR points into pixel-aligned multi-modal features, which we pass through a sequence of Transformer refinement stages.
  arXiv  Detail & Related papers  (2022-12-15T11:15:14Z)
• CRAT-Pred: Vehicle Trajectory Prediction with Crystal Graph Convolutional Neural Networks and Multi-Head Self-Attention [10.83642398981694]
  CRAT-Pred is a trajectory prediction model that does not rely on map information. The model achieves state-of-the-art performance with a significantly lower number of model parameters. In addition to that, we quantitatively show that the self-attention mechanism is able to learn social interactions between vehicles, with the weights representing a measurable interaction score.
  arXiv  Detail & Related papers  (2022-02-09T14:36:36Z)
• Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset [84.3946567650148]
  With over 100,000 scenes, each 20 seconds long at 10 Hz, our new dataset contains more than 570 hours of unique data over 1750 km of roadways. We use a high-accuracy 3D auto-labeling system to generate high quality 3D bounding boxes for each road agent. We introduce a new set of metrics that provides a comprehensive evaluation of both single agent and joint agent interaction motion forecasting models.
  arXiv  Detail & Related papers  (2021-04-20T17:19:05Z)
• Fine-Grained Vehicle Perception via 3D Part-Guided Visual Data Augmentation [77.60050239225086]
  We propose an effective training data generation process by fitting a 3D car model with dynamic parts to vehicles in real images. Our approach is fully automatic without any human interaction. We present a multi-task network for VUS parsing and a multi-stream network for VHI parsing.
  arXiv  Detail & Related papers  (2020-12-15T03:03:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.