Vision-Language Modeling with Regularized Spatial Transformer Networks for All Weather Crosswind Landing of Aircraft

• URL: http://arxiv.org/abs/2405.05574v2
• Date: Sat, 16 Nov 2024 14:49:53 GMT
• Title: Vision-Language Modeling with Regularized Spatial Transformer Networks for All Weather Crosswind Landing of Aircraft
• Authors: Debabrata Pal, Anvita Singh, Saumya Saumya, Shouvik Das,
• Abstract summary: A vision-based system trained to clear weather-induced visual degradations requires a robust landing dataset under various climatic conditions. We propose to synthesize harsh weather landing images by training a prompt-based climatic diffusion network. Also, we optimize a weather distillation model using a novel diffusion-distillation loss to learn to clear these visual degradations.
• Score: 0.3749861135832073
• License:
• Abstract: The intrinsic capability of the Human Vision System (HVS) to perceive depth of field and failure of Instrument Landing Systems (ILS) stimulates a pilot to perform a vision-based manual landing over an autoland approach. However, harsh weather creates challenges, and a pilot must have a clear view of runway elements before the minimum decision altitude. To aid in manual landing, a vision-based system trained to clear weather-induced visual degradations requires a robust landing dataset under various climatic conditions. Nevertheless, to acquire a dataset, flying an aircraft in dangerous weather impacts safety. Also, this system fails to generate reliable warnings, as localization of runway elements suffers from projective distortion while landing at crosswind. To combat, we propose to synthesize harsh weather landing images by training a prompt-based climatic diffusion network. Also, we optimize a weather distillation model using a novel diffusion-distillation loss to learn to clear these visual degradations. Precisely, the distillation model learns an inverse relationship with the diffusion network. Inference time, pre-trained distillation network directly clears weather-impacted onboard camera images, which can be further projected to display devices for improved visibility.Then, to tackle crosswind landing, a novel Regularized Spatial Transformer Networks (RuSTaN) module accurately warps landing images. It minimizes the localization error of runway object detector and helps generate reliable internal software warnings. Finally, we curated an aircraft landing dataset (AIRLAD) by simulating a landing scenario under various weather degradations and experimentally validated our contributions.

Related papers

• Power Line Aerial Image Restoration under dverse Weather: Datasets and Baselines [17.3009572002435]
  Power Line Autonomous Inspection (PLAI) plays a crucial role in the construction of smart grids. PLAI is completed by accurately detecting the electrical components and defects in the aerial images captured byUAVs. The visible quality of aerial images is inevitably degraded by adverse weather like haze, rain, or snow, which are found to drastically decrease the detection accuracy in our research. We propose a new task of Power Line Aerial Image Restoration under Adverse Weather (PLAIR-AW), which aims to recover clean and high-quality images from degraded images with bad weather.
  arXiv  Detail & Related papers  (2024-09-07T12:53:05Z)
• Real-Time Multi-Scene Visibility Enhancement for Promoting Navigational Safety of Vessels Under Complex Weather Conditions [48.529493393948435]
  The visible-light camera has emerged as an essential imaging sensor for marine surface vessels in intelligent waterborne transportation systems. The visual imaging quality inevitably suffers from several kinds of degradations under complex weather conditions. We develop a general-purpose multi-scene visibility enhancement method to restore degraded images captured under different weather conditions.
  arXiv  Detail & Related papers  (2024-09-02T23:46:27Z)
• Robust ADAS: Enhancing Robustness of Machine Learning-based Advanced Driver Assistance Systems for Adverse Weather [5.383130566626935]
  This paper employs a Denoising Deep Neural Network as a preprocessing step to transform adverse weather images into clear weather images. It improves driver visualization, which is critical for safe navigation in adverse weather conditions.
  arXiv  Detail & Related papers  (2024-07-02T18:03:52Z)
• Genuine Knowledge from Practice: Diffusion Test-Time Adaptation for Video Adverse Weather Removal [53.15046196592023]
  We introduce test-time adaptation into adverse weather removal in videos. We propose the first framework that integrates test-time adaptation into the iterative diffusion reverse process.
  arXiv  Detail & Related papers  (2024-03-12T14:21:30Z)
• Angle Robustness Unmanned Aerial Vehicle Navigation in GNSS-Denied Scenarios [66.05091704671503]
  We present a novel angle navigation paradigm to deal with flight deviation in point-to-point navigation tasks. We also propose a model that includes the Adaptive Feature Enhance Module, Cross-knowledge Attention-guided Module and Robust Task-oriented Head Module.
  arXiv  Detail & Related papers  (2024-02-04T08:41:20Z)
• Aircraft Landing Time Prediction with Deep Learning on Trajectory Images [18.536109188450876]
  In this study, a trajectory image-based deep learning method is proposed to predict ALTs for the aircraft entering the research airspace that covers the Terminal Maneuvering Area (TMA) The trajectory images contain various information, including the aircraft position, speed, heading, relative distances, and arrival traffic flows. We also use real-time runway usage obtained from the trajectory data and the external information such as aircraft types and weather conditions as additional inputs.
  arXiv  Detail & Related papers  (2024-01-02T07:56:05Z)
• Exploring the Application of Large-scale Pre-trained Models on Adverse Weather Removal [97.53040662243768]
  We propose a CLIP embedding module to make the network handle different weather conditions adaptively. This module integrates the sample specific weather prior extracted by CLIP image encoder together with the distribution specific information learned by a set of parameters.
  arXiv  Detail & Related papers  (2023-06-15T10:06:13Z)
• ScatterNeRF: Seeing Through Fog with Physically-Based Inverse Neural Rendering [83.75284107397003]
  We introduce ScatterNeRF, a neural rendering method which renders scenes and decomposes the fog-free background. We propose a disentangled representation for the scattering volume and the scene objects, and learn the scene reconstruction with physics-inspired losses. We validate our method by capturing multi-view In-the-Wild data and controlled captures in a large-scale fog chamber.
  arXiv  Detail & Related papers  (2023-05-03T13:24:06Z)
• Lidar Light Scattering Augmentation (LISA): Physics-based Simulation of Adverse Weather Conditions for 3D Object Detection [60.89616629421904]
  Lidar-based object detectors are critical parts of the 3D perception pipeline in autonomous navigation systems such as self-driving cars. They are sensitive to adverse weather conditions such as rain, snow and fog due to reduced signal-to-noise ratio (SNR) and signal-to-background ratio (SBR)
  arXiv  Detail & Related papers  (2021-07-14T21:10:47Z)
• Predicting Flight Delay with Spatio-Temporal Trajectory Convolutional Network and Airport Situational Awareness Map [20.579487904188802]
  We propose a vision-based solution to achieve a high forecasting accuracy, applicable to the airport. We propose an end-to-end deep learning architecture, TrajCNN, which captures both the spatial and temporal information from the situational awareness map. Our proposed framework obtained a good result (around 18 minutes error) for predicting flight departure delay at Los Angeles International Airport.
  arXiv  Detail & Related papers  (2021-05-19T07:38:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.