See&Say: Vision Language Guided Safe Zone Detection for Autonomous Package Delivery Drones

• URL: http://arxiv.org/abs/2604.13292v1
• Date: Tue, 14 Apr 2026 20:54:31 GMT
• Title: See&Say: Vision Language Guided Safe Zone Detection for Autonomous Package Delivery Drones
• Abstract summary: See&Say is a novel framework that combines geometric safety cues with semantic perception, guided by a Vision-Language Model (VLM) for iterative refinement.<n>We show that See&Say outperforms all baselines, achieving the highest accuracy and IoU for safety map prediction.<n>These findings highlight the promise of VLM-guided segmentation-depth fusion for advancing safe and practical drone-based package delivery.
• Score: 3.719121868494767
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Autonomous drone delivery systems are rapidly advancing, but ensuring safe and reliable package drop-offs remains highly challenging in cluttered urban and suburban environments where accurately identifying suitable package drop zones is critical. Existing approaches typically rely on either geometry-based analysis or semantic segmentation alone, but these methods lack the integrated semantic reasoning required for robust decision-making. To address this gap, we propose See&Say, a novel framework that combines geometric safety cues with semantic perception, guided by a Vision-Language Model (VLM) for iterative refinement. The system fuses monocular depth gradients with open-vocabulary detection masks to produce safety maps, while the VLM dynamically adjusts object category prompts and refines hazard detection across time, enabling reliable reasoning under dynamic conditions during the final delivery phase. When the primary drop-pad is occupied or unsafe, the proposed See&Say also identifies alternative candidate zones for package delivery. We curated a dataset of urban delivery scenarios with moving objects and human activities to evaluate the approach. Experimental results show that See&Say outperforms all baselines, achieving the highest accuracy and IoU for safety map prediction as well as superior performance in alternative drop zone evaluation across multiple thresholds. These findings highlight the promise of VLM-guided segmentation-depth fusion for advancing safe and practical drone-based package delivery.

Related papers

• DESCENT: Directed Edge Scene Encoding for Airport Surface Movement Prediction [54.94692733670454]
  DESCENT is a transformer-based architecture designed to handle heterogeneous dynamics and strict topological constraints.<n>Our approach features a Potential Reachable Set (PRS) context sampling mechanism that adaptively collects airfield environment context.<n>Our evaluations on the Amelia-10 benchmark demonstrate significant performance improvements over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-08-26T16:49:50Z)
• Exact, Efficient, and Safe Occlusion-Aware Planning Using AH-Polyhedrons [5.980069180948256]
  We propose APRO (AH-Polyhedron Reachability for Occlusions), an exact and efficient occlusion-aware planning framework.<n>Our key insight is to reformulate set-based safety conditions in prior work as unions of AH-polyhedrons.<n>We show how the resulting safety conditions can be integrated into optimization-based planners or a bisection search scheme for real-time applications.
  arXiv  Detail & Related papers  (2026-06-13T01:28:07Z)
• Evidence-Based Landing Site Selection and Vison-Based Landing for UAVs in Unstructured Environments [0.9236074230806578]
  This paper presents an evidence-based probabilistic framework for autonomous UAV landing.<n>It explicitly separates decision-making under uncertainty from execution via visual servoing.<n>It is validated through both real-world laboratory experiments and high-fidelity simulations in Nvidia Isaac Sim.
  arXiv  Detail & Related papers  (2026-05-02T13:05:20Z)
• Multi-Agent Off-World Exploration for Sparse Evidence Discovery via Gaussian Belief Mapping and Dual-Domain Coverage [15.524614683513834]
  We propose a multi-agent informative path planning framework for sparse evidence discovery based on Gaussian belief mapping and dual-domain coverage.<n> Experimental results in simulated lunar environments show that the proposed method consistently outperforms sampling-based and greedy baselines under different budgets and communication ranges.
  arXiv  Detail & Related papers  (2026-03-08T14:27:41Z)
• Eva-Tracker: ESDF-update-free, Visibility-aware Planning with Target Reacquisition for Robust Aerial Tracking [51.06229328990625]
  Eva-Tracker is a visibility-aware trajectory planning framework for aerial tracking.<n>It eliminates ESDF updates and incorporates a recovery-capable path generation method for target reacquisition.<n>Our approach delivers more robust tracking results with lower computational effort than existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-02-13T02:56:34Z)
• Agentic Spatio-Temporal Grounding via Collaborative Reasoning [80.83158605034465]
  Temporal Video Grounding aims to retrieve thetemporal tube of a target object or person in a video given a text query.<n>We propose the Agentic Spatio-Temporal Grounder (ASTG) framework for the task of STVG towards an open-world and training-free scenario.<n>Specifically, two specialized agents SRA (Spatial Reasoning Agent) and TRA (Temporal Reasoning Agent) constructed leveraging on modern Multimoal Large Language Models (MLLMs)<n>Experiments on popular benchmarks demonstrate the superiority of the proposed approach where it outperforms existing weakly-supervised and zero-shot approaches by a margin
  arXiv  Detail & Related papers  (2026-02-10T10:16:27Z)
• Semantically Aware UAV Landing Site Assessment from Remote Sensing Imagery via Multimodal Large Language Models [5.987458168544856]
  Safe UAV emergency landing requires understanding complex semantic risks invisible to traditional geometric sensors.<n>We propose a novel framework leveraging Remote Sensing (RS) imagery and Multimodal Large Language Models (MLLMs) for context-aware landing site assessment.
  arXiv  Detail & Related papers  (2026-02-01T11:30:03Z)
• OS-Sentinel: Towards Safety-Enhanced Mobile GUI Agents via Hybrid Validation in Realistic Workflows [77.95511352806261]
  Computer-using agents powered by Vision-Language Models (VLMs) have demonstrated human-like capabilities in operating digital environments like mobile platforms.<n>We propose OS-Sentinel, a novel hybrid safety detection framework that combines a Formal Verifier for detecting explicit system-level violations with a Contextual Judge for assessing contextual risks and agent actions.
  arXiv  Detail & Related papers  (2025-10-28T13:22:39Z)
• Rethinking Safety in LLM Fine-tuning: An Optimization Perspective [56.31306558218838]
  We show that poor optimization choices, rather than inherent trade-offs, often cause safety problems, measured as harmful responses to adversarial prompts.<n>We propose a simple exponential moving average (EMA) momentum technique in parameter space that preserves safety performance.<n>Our experiments on the Llama families across multiple datasets demonstrate that safety problems can largely be avoided without specialized interventions.
  arXiv  Detail & Related papers  (2025-08-17T23:46:36Z)
• NOVA: Navigation via Object-Centric Visual Autonomy for High-Speed Target Tracking in Unstructured GPS-Denied Environments [56.35569661650558]
  We introduce NOVA, a fully onboard, object-centric framework that enables robust target tracking and collision-aware navigation.<n>Rather than constructing a global map, NOVA formulates perception, estimation, and control entirely in the target's reference frame.<n>We validate NOVA across challenging real-world scenarios, including urban mazes, forest trails, and repeated transitions through buildings with intermittent GPS loss.
  arXiv  Detail & Related papers  (2025-06-23T14:28:30Z)
• VisLanding: Monocular 3D Perception for UAV Safe Landing via Depth-Normal Synergy [0.0]
  VisLanding is a perception-based framework for safe UAV (Unmanned Aerial Vehicle) landing.<n>The proposed method exhibits superior generalization and robustness in cross-domain testing compared to other approaches.
  arXiv  Detail & Related papers  (2025-06-17T13:51:16Z)
• From Shadows to Safety: Occlusion Tracking and Risk Mitigation for Urban Autonomous Driving [1.8434042562191815]
  This research builds upon and extends existing approaches in risk-aware motion planning and occlusion tracking.<n>We enhance a phantom agent-centric model by incorporating sequential reasoning to track occluded areas and predict potential hazards.<n> Simulations demonstrate that the proposed approach improves situational awareness and balances proactive safety with efficient traffic flow.
  arXiv  Detail & Related papers  (2025-04-02T06:48:50Z)
• Deep Monocular Hazard Detection for Safe Small Body Landing [12.922946578413578]
  Hazard detection and avoidance is a key technology for future robotic small body sample return and lander missions. We propose a novel safety mapping paradigm that leverages deep semantic segmentation techniques to predict landing safety directly from a single monocular image. We demonstrate precise and accurate safety mapping performance on real in-situ imagery of prospective sample sites from the OSIRIS-REx mission.
  arXiv  Detail & Related papers  (2023-01-30T19:40:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.