ASMA: An Adaptive Safety Margin Algorithm for Vision-Language Drone Navigation via Scene-Aware Control Barrier Functions

• URL: http://arxiv.org/abs/2409.10283v1
• Date: Mon, 16 Sep 2024 13:44:50 GMT
• Title: ASMA: An Adaptive Safety Margin Algorithm for Vision-Language Drone Navigation via Scene-Aware Control Barrier Functions
• Authors: Sourav Sanyal, Kaushik Roy,
• Abstract summary: Control barrier functions (CBFs) are efficient tools which guarantee safety by solving an optimal control problem. We formulate a novel scene-aware CBF using ego-centric observations obtained through an RGB-D sensor. We propose ASMA -- an Adaptive Safety Margin -- that crops the drone's depth map for tracking moving object(s) to perform scene-aware CBF evaluation on-the-fly.
• Score: 9.645098673995317
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In the rapidly evolving field of vision-language navigation (VLN), ensuring robust safety mechanisms remains an open challenge. Control barrier functions (CBFs) are efficient tools which guarantee safety by solving an optimal control problem. In this work, we consider the case of a teleoperated drone in a VLN setting, and add safety features by formulating a novel scene-aware CBF using ego-centric observations obtained through an RGB-D sensor. As a baseline, we implement a vision-language understanding module which uses the contrastive language image pretraining (CLIP) model to query about a user-specified (in natural language) landmark. Using the YOLO (You Only Look Once) object detector, the CLIP model is queried for verifying the cropped landmark, triggering downstream navigation. To improve navigation safety of the baseline, we propose ASMA -- an Adaptive Safety Margin Algorithm -- that crops the drone's depth map for tracking moving object(s) to perform scene-aware CBF evaluation on-the-fly. By identifying potential risky observations from the scene, ASMA enables real-time adaptation to unpredictable environmental conditions, ensuring optimal safety bounds on a VLN-powered drone actions. Using the robot operating system (ROS) middleware on a parrot bebop2 quadrotor in the gazebo environment, ASMA offers 59.4% - 61.8% increase in success rates with insignificant 5.4% - 8.2% increases in trajectory lengths compared to the baseline CBF-less VLN while recovering from unsafe situations.

Related papers

• Low-altitude Friendly-Jamming for Satellite-Maritime Communications via Generative AI-enabled Deep Reinforcement Learning [72.72954660774002]
  Low Earth Orbit (LEO) satellites can be used to assist maritime wireless communications for data transmission across wide-ranging areas. Extensive coverage of LEO satellites, combined with openness of channels, can cause the communication process to suffer from security risks. This paper presents a low-altitude friendly-jamming LEO satellite-maritime communication system enabled by a unmanned aerial vehicle.
  arXiv  Detail & Related papers  (2025-01-26T10:13:51Z)
• One Filter to Deploy Them All: Robust Safety for Quadrupedal Navigation in Unknown Environments [10.103090762559708]
  We propose an observation-conditioned reachability-based (OCR) safety-filter framework. Our key idea is to use an OCR value network (OCR-VN) that predicts the optimal control-theoretic safety value function for new failure regions. We demonstrate that the proposed framework can automatically safeguard a wide range of hierarchical quadruped controllers.
  arXiv  Detail & Related papers  (2024-12-13T09:21:02Z)
• A Cross-Scene Benchmark for Open-World Drone Active Tracking [54.235808061746525]
  Drone Visual Active Tracking aims to autonomously follow a target object by controlling the motion system based on visual observations. We propose a unified cross-scene cross-domain benchmark for open-world drone active tracking called DAT. We also propose a reinforcement learning-based drone tracking method called R-VAT.
  arXiv  Detail & Related papers  (2024-12-01T09:37:46Z)
• Towards Building Secure UAV Navigation with FHE-aware Knowledge Distillation [0.0]
  This paper proposes an innovative approach that leverages Knowledge Distillation to enhance the practicality of secure UAV navigation. By integrating RL and FHE, our framework addresses vulnerabilities to adversarial attacks while enabling real-time processing of encrypted UAV camera feeds. To mitigate FHE's latency, Knowledge Distillation is employed to compress the network, resulting in an impressive 18x speedup without compromising performance.
  arXiv  Detail & Related papers  (2024-11-01T07:04:24Z)
• Long-Range Vision-Based UAV-assisted Localization for Unmanned Surface Vehicles [7.384309568198598]
  Global positioning system (GPS) has become an indispensable navigation method for field operations with unmanned surface vehicles (USVs) in marine environments. GPS may not always be available outdoors because it is vulnerable to natural interference and malicious jamming attacks. We present a novel method that utilizes an Unmanned Aerial Vehicle (UAV) to assist in localizing USVs in restricted marine environments.
  arXiv  Detail & Related papers  (2024-08-21T08:37:37Z)
• Secure Navigation using Landmark-based Localization in a GPS-denied Environment [1.19658449368018]
  This paper proposes a novel framework that integrates landmark-based localization (LanBLoc) with an Extended Kalman Filter (EKF) to predict the future state of moving entities along the battlefield. We present a simulated battlefield scenario for two different approaches that guide a moving entity through an obstacle and hazard-free path.
  arXiv  Detail & Related papers  (2024-02-22T04:41:56Z)
• Angle Robustness Unmanned Aerial Vehicle Navigation in GNSS-Denied Scenarios [66.05091704671503]
  We present a novel angle navigation paradigm to deal with flight deviation in point-to-point navigation tasks. We also propose a model that includes the Adaptive Feature Enhance Module, Cross-knowledge Attention-guided Module and Robust Task-oriented Head Module.
  arXiv  Detail & Related papers  (2024-02-04T08:41:20Z)
• Segmentation of Drone Collision Hazards in Airborne RADAR Point Clouds Using PointNet [0.7067443325368975]
  A critical prerequisite for the integration is equipping UAVs with enhanced situational awareness to ensure safe operations. Our study leverages radar technology for novel end-to-end semantic segmentation of aerial point clouds to simultaneously identify multiple collision hazards. To our knowledge, this is the first approach addressing simultaneous identification of multiple collision threats in an aerial setting, achieving a robust 94% accuracy.
  arXiv  Detail & Related papers  (2023-11-06T16:04:58Z)
• Meta-Learning Priors for Safe Bayesian Optimization [72.8349503901712]
  We build on a meta-learning algorithm, F-PACOH, capable of providing reliable uncertainty quantification in settings of data scarcity. As core contribution, we develop a novel framework for choosing safety-compliant priors in a data-riven manner. On benchmark functions and a high-precision motion system, we demonstrate that our meta-learned priors accelerate the convergence of safe BO approaches.
  arXiv  Detail & Related papers  (2022-10-03T08:38:38Z)
• BarrierNet: A Safety-Guaranteed Layer for Neural Networks [50.86816322277293]
  BarrierNet allows the safety constraints of a neural controller be adaptable to changing environments. We evaluate them on a series of control problems such as traffic merging and robot navigations in 2D and 3D space.
  arXiv  Detail & Related papers  (2021-11-22T15:38:11Z)
• Transferable Deep Reinforcement Learning Framework for Autonomous Vehicles with Joint Radar-Data Communications [69.24726496448713]
  We propose an intelligent optimization framework based on the Markov Decision Process (MDP) to help the AV make optimal decisions. We then develop an effective learning algorithm leveraging recent advances of deep reinforcement learning techniques to find the optimal policy for the AV. We show that the proposed transferable deep reinforcement learning framework reduces the obstacle miss detection probability by the AV up to 67% compared to other conventional deep reinforcement learning approaches.
  arXiv  Detail & Related papers  (2021-05-28T08:45:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.