Vision-based Perception System for Automated Delivery Robot-Pedestrians Interactions

• URL: http://arxiv.org/abs/2508.03541v1
• Date: Tue, 05 Aug 2025 15:10:09 GMT
• Title: Vision-based Perception System for Automated Delivery Robot-Pedestrians Interactions
• Abstract summary: We develop the complete pipeline for a single vision sensor based multi-pedestrian detection and tracking, pose estimation, and monocular depth perception.<n>Results show measurable improvements, including up to a 10% increase in identity preservation.<n> Notably, the system identifies vulnerable pedestrian groups supporting more socially aware and inclusive robot behaviour.
• Score: 4.378407481656902
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The integration of Automated Delivery Robots (ADRs) into pedestrian-heavy urban spaces introduces unique challenges in terms of safe, efficient, and socially acceptable navigation. We develop the complete pipeline for a single vision sensor based multi-pedestrian detection and tracking, pose estimation, and monocular depth perception. Leveraging the real-world MOT17 dataset sequences, this study demonstrates how integrating human-pose estimation and depth cues enhances pedestrian trajectory prediction and identity maintenance, even under occlusions and dense crowds. Results show measurable improvements, including up to a 10% increase in identity preservation (IDF1), a 7% improvement in multiobject tracking accuracy (MOTA), and consistently high detection precision exceeding 85%, even in challenging scenarios. Notably, the system identifies vulnerable pedestrian groups supporting more socially aware and inclusive robot behaviour.

Related papers

• Human-Aware Target Tracking and Navigation: Fusing Kinematic State Estimation with Structural Map Constraints [3.9308813646788625]
  This research presents an end-to-end autonomous navigation stack that addresses target occlusion through map-informed spatial reasoning.<n>The proposed system features a multi-modal perception pipeline, fusing deep learning-based visual tracking with 2-dimensional LiDAR point clustering to maintain high-fidelity tracking of a tagged person.<n>Real-world evaluations in dense multi-person environments demonstrate the system's robustness, achieving a 71.4% target reacquisition success rate.
  arXiv  Detail & Related papers  (2026-09-07T06:31:17Z)
• Face versus Body Tracking for Human-Robot Interaction: An Egocentric Dataset [7.3852311170912905]
  State-of-the-art computer vision models are heavily optimized for surveillance or autonomous driving.<n>A social robot faces distinct egocentric challenges, such as humans bouncing, obstructing each other, or leaving the frame.<n>We introduce a novel, custom-annotated egocentric dataset collected via the Furhat robot to capture complex social dynamics.
  arXiv  Detail & Related papers  (2026-06-02T14:15:17Z)
• ManipArena: Comprehensive Real-world Evaluation of Reasoning-Oriented Generalist Robot Manipulation [66.02142169323521]
  Vision-Language-Action models and world models have recently emerged as promising paradigms for general-purpose robotic intelligence.<n>Existing benchmarks are largely simulator-centric, which provide controllability but fail to capture the reality gap caused by perception noise.<n>We introduce ManipArena, a standardized evaluation framework designed to bridge simulation and real-world execution.
  arXiv  Detail & Related papers  (2026-03-30T15:06:41Z)
• RoboTracer: Mastering Spatial Trace with Reasoning in Vision-Language Models for Robotics [53.053660003572965]
  We propose RoboTracer, a 3D-aware VLM that first achieves both 3D spatial referring and measuring.<n>RoboTracer advances multi-step metric-grounded reasoning via reinforcement fine-tuning.<n>We present TraceSpatial-Bench, a challenging benchmark to evaluate spatial tracing.
  arXiv  Detail & Related papers  (2025-12-15T18:52:43Z)
• AerialMind: Towards Referring Multi-Object Tracking in UAV Scenarios [64.51320327698231]
  We introduce AerialMind, the first large-scale RMOT benchmark in UAV scenarios.<n>We develop an innovative semi-automated collaborative agent-based labeling assistant framework.<n>We also propose HawkEyeTrack, a novel method that collaboratively enhances vision-language representation learning.
  arXiv  Detail & Related papers  (2025-11-26T04:44:27Z)
• Progressive Bird's Eye View Perception for Safety-Critical Autonomous Driving: A Comprehensive Survey [20.7823289124196]
  Bird's-Eye-View (BEV) perception has become a foundational paradigm in autonomous driving.<n>This survey provides the first comprehensive review of BEV perception from a safety-critical perspective.
  arXiv  Detail & Related papers  (2025-08-11T02:40:46Z)
• Privacy-Preserving Driver Drowsiness Detection with Spatial Self-Attention and Federated Learning [10.067641629547014]
  Driver drowsiness is one of the main causes of road accidents and is recognized as a leading contributor to traffic-related fatalities.<n>We propose a novel framework for drowsiness detection that is designed to work effectively with heterogeneous and decentralized data.
  arXiv  Detail & Related papers  (2025-08-01T03:12:01Z)
• YOLO-APD: Enhancing YOLOv8 for Robust Pedestrian Detection on Complex Road Geometries [0.0]
  This paper introduces YOLO-APD, a novel deep learning architecture enhancing the YOLOv8 framework specifically for this challenge.<n>YOLO-APD achieves state-of-the-art detection accuracy, reaching 77.7% mAP@0.5:0.95 and exceptional pedestrian recall exceeding 96%.<n>It maintains real-time processing capabilities at 100 FPS, showcasing a superior balance between accuracy and efficiency.
  arXiv  Detail & Related papers  (2025-07-07T18:03:40Z)
• Enhancing Traffic Sign Recognition On The Performance Based On Yolov8 [0.0]
  This thesis presents an enhanced YOLOv8-based detection system that integrates advanced data augmentation techniques.<n>Experiments conducted on datasets including GTSRB, TT100K, and GTSDB demonstrate marked improvements in detection accuracy, robustness under adverse conditions, and real-time inference on edge devices.
  arXiv  Detail & Related papers  (2025-04-02T07:28:05Z)
• A Cross-Scene Benchmark for Open-World Drone Active Tracking [54.235808061746525]
  Drone Visual Active Tracking aims to autonomously follow a target object by controlling the motion system based on visual observations.<n>We propose a unified cross-scene cross-domain benchmark for open-world drone active tracking called DAT.<n>We also propose a reinforcement learning-based drone tracking method called R-VAT.
  arXiv  Detail & Related papers  (2024-12-01T09:37:46Z)
• Traffic and Safety Rule Compliance of Humans in Diverse Driving Situations [48.924085579865334]
  Analyzing human data is crucial for developing autonomous systems that replicate safe driving practices. This paper presents a comparative evaluation of human compliance with traffic and safety rules across multiple trajectory prediction datasets.
  arXiv  Detail & Related papers  (2024-11-04T09:21:00Z)
• RoboSense: Large-scale Dataset and Benchmark for Egocentric Robot Perception and Navigation in Crowded and Unstructured Environments [62.5830455357187]
  We setup an egocentric multi-sensor data collection platform based on 3 main types of sensors (Camera, LiDAR and Fisheye)<n>A large-scale multimodal dataset is constructed, named RoboSense, to facilitate egocentric robot perception.
  arXiv  Detail & Related papers  (2024-08-28T03:17:40Z)
• Real-Time Detection and Analysis of Vehicles and Pedestrians using Deep Learning [0.0]
  Current traffic monitoring systems confront major difficulty in recognizing small objects and pedestrians effectively in real-time. Our project focuses on the creation and validation of an advanced deep-learning framework capable of processing complex visual input for precise, real-time recognition of cars and people. The YOLOv8 Large version proved to be the most effective, especially in pedestrian recognition, with great precision and robustness.
  arXiv  Detail & Related papers  (2024-04-11T18:42:14Z)
• SHIFT: A Synthetic Driving Dataset for Continuous Multi-Task Domain Adaptation [152.60469768559878]
  SHIFT is the largest multi-task synthetic dataset for autonomous driving. It presents discrete and continuous shifts in cloudiness, rain and fog intensity, time of day, and vehicle and pedestrian density. Our dataset and benchmark toolkit are publicly available at www.vis.xyz/shift.
  arXiv  Detail & Related papers  (2022-06-16T17:59:52Z)
• Multi-Robot Collaborative Perception with Graph Neural Networks [6.383576104583731]
  We propose a general-purpose Graph Neural Network (GNN) with the main goal to increase, in multi-robot perception tasks. We show that the proposed framework can address multi-view visual perception problems such as monocular depth estimation and semantic segmentation.
  arXiv  Detail & Related papers  (2022-01-05T18:47:07Z)
• Benchmarking high-fidelity pedestrian tracking systems for research, real-time monitoring and crowd control [55.41644538483948]
  High-fidelity pedestrian tracking in real-life conditions has been an important tool in fundamental crowd dynamics research. As this technology advances, it is becoming increasingly useful also in society. To successfully employ pedestrian tracking techniques in research and technology, it is crucial to validate and benchmark them for accuracy. We present and discuss a benchmark suite, towards an open standard in the community, for privacy-respectful pedestrian tracking techniques.
  arXiv  Detail & Related papers  (2021-08-26T11:45:26Z)
• TRiPOD: Human Trajectory and Pose Dynamics Forecasting in the Wild [77.59069361196404]
  TRiPOD is a novel method for predicting body dynamics based on graph attentional networks. To incorporate a real-world challenge, we learn an indicator representing whether an estimated body joint is visible/invisible at each frame. Our evaluation shows that TRiPOD outperforms all prior work and state-of-the-art specifically designed for each of the trajectory and pose forecasting tasks.
  arXiv  Detail & Related papers  (2021-04-08T20:01:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.