A Pedestrian-Vehicle Interaction Benchmark and Annotation Framework for Unstructured Scenes via Uncalibrated Cameras

• URL: http://arxiv.org/abs/2605.25947v1
• Date: Mon, 25 May 2026 15:26:18 GMT
• Title: A Pedestrian-Vehicle Interaction Benchmark and Annotation Framework for Unstructured Scenes via Uncalibrated Cameras
• Abstract summary: We propose a dataset based on video data from uncalibrated surveillance cameras and present PINNS (Pedestrian-vehicle Interaction dataset from uNcalibrated cameras in uNstructured Scenes)<n>The dataset covers multiple countries and regions, includes diverse typical traffic scenarios, and considers variations in seasons, lighting conditions, and weather.<n>We hope our framework and dataset will facilitate research on trajectory prediction and autonomous driving in complex mixed traffic scenarios.
• Score: 14.16993152950793
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Predicting the interaction between pedestrian and vehicle is essential for autonomous driving safety in unstructured and semi-structured scenarios; however, this task is severely hindered by the scarcity of public datasets that feature dense pedestrian-vehicle interactions. Most current studies rely on structured road data, leaving the complex, heterogeneous interactions found in unstructured environments insufficiently represented and researched. In this paper, we propose a dataset annotation framework based on video data from uncalibrated surveillance cameras and present PINNS (Pedestrian-vehicle Interaction dataset from uNcalibrated cameras in uNstructured Scenes). The dataset covers multiple countries and regions, includes diverse typical traffic scenarios, and considers variations in seasons, lighting conditions, and weather. It focuses on complex scenes with dense pedestrian-vehicle interactions and is designed to be easily extensible. The dataset is constructed and annotated according to the standard issued by the Chinese Association of Automation, providing both trajectory data and corresponding scene-level information. Furthermore, this paper analyzes current challenges and research directions in heterogeneous agent trajectory prediction, shows the necessity and usefulness of the proposed dataset. We hope our framework and dataset will facilitate research on trajectory prediction and autonomous driving in complex mixed traffic scenarios. PINNS is publicly available at https://github.com/Songan-Lab.

Related papers

• HetroD: A High-Fidelity Drone Dataset and Benchmark for Autonomous Driving in Heterogeneous Traffic [49.31491001465465]
  HetroD is a dataset and benchmark for developing autonomous driving systems in heterogeneous environments.<n>HetroD targets the critical challenge of navi- gating real-world heterogeneous traffic dominated by vulner- able road users (VRUs)
  arXiv  Detail & Related papers  (2026-02-03T12:12:47Z)
• Context-based Motion Retrieval using Open Vocabulary Methods for Autonomous Driving [0.5249805590164902]
  We propose a novel context-aware motion retrieval framework to support targeted evaluation of autonomous driving systems in diverse, human-centered scenarios.<n>Our approach outperforms state-of-the-art models by up to 27.5% accuracy in motion-context retrieval, when evaluated on the WayMoCo dataset.
  arXiv  Detail & Related papers  (2025-08-01T12:41:52Z)
• Pedestrian Intention and Trajectory Prediction in Unstructured Traffic Using IDD-PeD [26.011293248078797]
  We introduce an Indian driving pedestrian dataset designed to address the complexities of modeling pedestrian behavior in unstructured environments.<n>The dataset provides high-level and detailed low-level comprehensive annotations focused on pedestrians requiring the ego-vehicle's attention.
  arXiv  Detail & Related papers  (2025-06-27T10:41:18Z)
• PFSD: A Multi-Modal Pedestrian-Focus Scene Dataset for Rich Tasks in Semi-Structured Environments [73.80718037070773]
  We present the multi-modal Pedestrian-Focused Scene dataset, rigorously annotated in semi-structured scenes with the format of nuScenes.<n>We also propose a novel Hybrid Multi-Scale Fusion Network (HMFN) to detect pedestrians in densely populated and occluded scenarios.
  arXiv  Detail & Related papers  (2025-02-21T09:57:53Z)
• ROAD-Waymo: Action Awareness at Scale for Autonomous Driving [17.531603453254434]
  ROAD-Waymo is an extensive dataset for the development and benchmarking of techniques for agent, action, location and event detection in road scenes. Considerably larger and more challenging than any existing dataset (and encompassing multiple cities), it comes with 198k annotated video frames, 54k agent tubes, 3.9M bounding boxes and a total of 12.4M labels.
  arXiv  Detail & Related papers  (2024-11-03T20:46:50Z)
• JRDB-Traj: A Dataset and Benchmark for Trajectory Forecasting in Crowds [79.00975648564483]
  Trajectory forecasting models, employed in fields such as robotics, autonomous vehicles, and navigation, face challenges in real-world scenarios. This dataset provides comprehensive data, including the locations of all agents, scene images, and point clouds, all from the robot's perspective. The objective is to predict the future positions of agents relative to the robot using raw sensory input data.
  arXiv  Detail & Related papers  (2023-11-05T18:59:31Z)
• Graph Convolutional Networks for Complex Traffic Scenario Classification [0.7919810878571297]
  A scenario-based testing approach can reduce the time required to obtain statistically significant evidence of the safety of Automated Driving Systems. Most methods on scenario classification do not work for complex scenarios with diverse environments. We propose a method for complex traffic scenario classification that is able to model the interaction of a vehicle with the environment.
  arXiv  Detail & Related papers  (2023-10-26T20:51:24Z)
• IDD-3D: Indian Driving Dataset for 3D Unstructured Road Scenes [79.18349050238413]
  Preparation and training of deploy-able deep learning architectures require the models to be suited to different traffic scenarios. An unstructured and complex driving layout found in several developing countries such as India poses a challenge to these models. We build a new dataset, IDD-3D, which consists of multi-modal data from multiple cameras and LiDAR sensors with 12k annotated driving LiDAR frames.
  arXiv  Detail & Related papers  (2022-10-23T23:03:17Z)
• Decentralized Vehicle Coordination: The Berkeley DeepDrive Drone Dataset and Consensus-Based Models [76.32775745488073]
  We present a novel dataset and modeling framework designed to study motion planning in understructured environments.<n>We demonstrate that a consensus-based modeling approach can effectively explain the emergence of priority orders observed in our dataset.
  arXiv  Detail & Related papers  (2022-09-19T05:06:57Z)
• Euro-PVI: Pedestrian Vehicle Interactions in Dense Urban Centers [126.81938540470847]
  We propose Euro-PVI, a dataset of pedestrian and bicyclist trajectories. In this work, we develop a joint inference model that learns an expressive multi-modal shared latent space across agents in the urban scene. We achieve state of the art results on the nuScenes and Euro-PVI datasets demonstrating the importance of capturing interactions between ego-vehicle and pedestrians (bicyclists) for accurate predictions.
  arXiv  Detail & Related papers  (2021-06-22T15:40:21Z)
• Predicting Vehicles Trajectories in Urban Scenarios with Transformer Networks and Augmented Information [0.0]
  This paper exploits simple structures for predicting pedestrian trajectories, based on Transformer Networks. We adapt their use to the problem of vehicle trajectory prediction in urban scenarios in prediction horizons up to 5 seconds. Our model achieves state-of-the-art results and proves to be flexible and adaptable to different types of urban contexts.
  arXiv  Detail & Related papers  (2021-06-01T15:18:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.