Vision-Language Modeling with Regularized Spatial Transformer Networks for All Weather Crosswind Landing of Aircraft

• URL: http://arxiv.org/abs/2405.05574v1
• Date: Thu, 9 May 2024 06:48:42 GMT
• Title: Vision-Language Modeling with Regularized Spatial Transformer Networks for All Weather Crosswind Landing of Aircraft
• Authors: Debabrata Pal, Anvita Singh, Saumya Saumya, Shouvik Das,
• Abstract summary: In harsh weather, a pilot must have a clear view of runway elements before the minimum decision altitude. A vision-based system tailored to localize runway elements likewise gets affected, especially during crosswind. We propose to integrate a prompt-based climatic diffusion network with a weather distillation model using a novel diffusion-distillation loss.
• Score: 0.3749861135832073
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The intrinsic capability to perceive depth of field and extract salient information by the Human Vision System (HVS) stimulates a pilot to perform manual landing over an autoland approach. However, harsh weather creates visibility hindrances, and a pilot must have a clear view of runway elements before the minimum decision altitude. To help a pilot in manual landing, a vision-based system tailored to localize runway elements likewise gets affected, especially during crosswind due to the projective distortion of aircraft camera images. To combat this, we propose to integrate a prompt-based climatic diffusion network with a weather distillation model using a novel diffusion-distillation loss. Precisely, the diffusion model synthesizes climatic-conditioned landing images, and the weather distillation model learns inverse mapping by clearing those visual degradations. Then, to tackle the crosswind landing scenario, a novel Regularized Spatial Transformer Networks (RuSTaN) learns to accurately calibrate for projective distortion using self-supervised learning, which minimizes localization error by the downstream runway object detector. Finally, we have simulated a clear-day landing scenario at the busiest airport globally to curate an image-based Aircraft Landing Dataset (AIRLAD) and experimentally validated our contributions using this dataset to benchmark the performance.

Related papers

• Power Line Aerial Image Restoration under dverse Weather: Datasets and Baselines [17.3009572002435]
  Power Line Autonomous Inspection (PLAI) plays a crucial role in the construction of smart grids. PLAI is completed by accurately detecting the electrical components and defects in the aerial images captured byUAVs. The visible quality of aerial images is inevitably degraded by adverse weather like haze, rain, or snow, which are found to drastically decrease the detection accuracy in our research. We propose a new task of Power Line Aerial Image Restoration under Adverse Weather (PLAIR-AW), which aims to recover clean and high-quality images from degraded images with bad weather.
  arXiv  Detail & Related papers  (2024-09-07T12:53:05Z)
• Real-Time Multi-Scene Visibility Enhancement for Promoting Navigational Safety of Vessels Under Complex Weather Conditions [48.529493393948435]
  The visible-light camera has emerged as an essential imaging sensor for marine surface vessels in intelligent waterborne transportation systems. The visual imaging quality inevitably suffers from several kinds of degradations under complex weather conditions. We develop a general-purpose multi-scene visibility enhancement method to restore degraded images captured under different weather conditions.
  arXiv  Detail & Related papers  (2024-09-02T23:46:27Z)
• Robust ADAS: Enhancing Robustness of Machine Learning-based Advanced Driver Assistance Systems for Adverse Weather [5.383130566626935]
  This paper employs a Denoising Deep Neural Network as a preprocessing step to transform adverse weather images into clear weather images. It improves driver visualization, which is critical for safe navigation in adverse weather conditions.
  arXiv  Detail & Related papers  (2024-07-02T18:03:52Z)
• Genuine Knowledge from Practice: Diffusion Test-Time Adaptation for Video Adverse Weather Removal [53.15046196592023]
  We introduce test-time adaptation into adverse weather removal in videos. We propose the first framework that integrates test-time adaptation into the iterative diffusion reverse process.
  arXiv  Detail & Related papers  (2024-03-12T14:21:30Z)
• Angle Robustness Unmanned Aerial Vehicle Navigation in GNSS-Denied Scenarios [66.05091704671503]
  We present a novel angle navigation paradigm to deal with flight deviation in point-to-point navigation tasks. We also propose a model that includes the Adaptive Feature Enhance Module, Cross-knowledge Attention-guided Module and Robust Task-oriented Head Module.
  arXiv  Detail & Related papers  (2024-02-04T08:41:20Z)
• Aircraft Landing Time Prediction with Deep Learning on Trajectory Images [18.536109188450876]
  In this study, a trajectory image-based deep learning method is proposed to predict ALTs for the aircraft entering the research airspace that covers the Terminal Maneuvering Area (TMA) The trajectory images contain various information, including the aircraft position, speed, heading, relative distances, and arrival traffic flows. We also use real-time runway usage obtained from the trajectory data and the external information such as aircraft types and weather conditions as additional inputs.
  arXiv  Detail & Related papers  (2024-01-02T07:56:05Z)
• Exploring the Application of Large-scale Pre-trained Models on Adverse Weather Removal [97.53040662243768]
  We propose a CLIP embedding module to make the network handle different weather conditions adaptively. This module integrates the sample specific weather prior extracted by CLIP image encoder together with the distribution specific information learned by a set of parameters.
  arXiv  Detail & Related papers  (2023-06-15T10:06:13Z)
• ScatterNeRF: Seeing Through Fog with Physically-Based Inverse Neural Rendering [83.75284107397003]
  We introduce ScatterNeRF, a neural rendering method which renders scenes and decomposes the fog-free background. We propose a disentangled representation for the scattering volume and the scene objects, and learn the scene reconstruction with physics-inspired losses. We validate our method by capturing multi-view In-the-Wild data and controlled captures in a large-scale fog chamber.
  arXiv  Detail & Related papers  (2023-05-03T13:24:06Z)
• Lidar Light Scattering Augmentation (LISA): Physics-based Simulation of Adverse Weather Conditions for 3D Object Detection [60.89616629421904]
  Lidar-based object detectors are critical parts of the 3D perception pipeline in autonomous navigation systems such as self-driving cars. They are sensitive to adverse weather conditions such as rain, snow and fog due to reduced signal-to-noise ratio (SNR) and signal-to-background ratio (SBR)
  arXiv  Detail & Related papers  (2021-07-14T21:10:47Z)
• Predicting Flight Delay with Spatio-Temporal Trajectory Convolutional Network and Airport Situational Awareness Map [20.579487904188802]
  We propose a vision-based solution to achieve a high forecasting accuracy, applicable to the airport. We propose an end-to-end deep learning architecture, TrajCNN, which captures both the spatial and temporal information from the situational awareness map. Our proposed framework obtained a good result (around 18 minutes error) for predicting flight departure delay at Los Angeles International Airport.
  arXiv  Detail & Related papers  (2021-05-19T07:38:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.