Spatio-Temporal Turbulence Mitigation: A Translational Perspective

• URL: http://arxiv.org/abs/2401.04244v2
• Date: Sun, 7 Apr 2024 20:13:45 GMT
• Title: Spatio-Temporal Turbulence Mitigation: A Translational Perspective
• Authors: Xingguang Zhang, Nicholas Chimitt, Yiheng Chi, Zhiyuan Mao, Stanley H. Chan,
• Abstract summary: We present the Deep Atmospheric TUrbulence Mitigation network ( DATUM) DATUM aims to overcome major challenges when transitioning from classical to deep learning approaches. A large-scale training dataset, ATSyn, is presented as a co-invention to enable generalization in real turbulence.
• Score: 13.978156774471744
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recovering images distorted by atmospheric turbulence is a challenging inverse problem due to the stochastic nature of turbulence. Although numerous turbulence mitigation (TM) algorithms have been proposed, their efficiency and generalization to real-world dynamic scenarios remain severely limited. Building upon the intuitions of classical TM algorithms, we present the Deep Atmospheric TUrbulence Mitigation network (DATUM). DATUM aims to overcome major challenges when transitioning from classical to deep learning approaches. By carefully integrating the merits of classical multi-frame TM methods into a deep network structure, we demonstrate that DATUM can efficiently perform long-range temporal aggregation using a recurrent fashion, while deformable attention and temporal-channel attention seamlessly facilitate pixel registration and lucky imaging. With additional supervision, tilt and blur degradation can be jointly mitigated. These inductive biases empower DATUM to significantly outperform existing methods while delivering a tenfold increase in processing speed. A large-scale training dataset, ATSyn, is presented as a co-invention to enable generalization in real turbulence. Our code and datasets are available at https://xg416.github.io/DATUM.

Related papers

• Long-range Turbulence Mitigation: A Large-scale Dataset and A Coarse-to-fine Framework [12.869617522032986]
  We construct a large-scale real long-range atmospheric turbulence dataset (RLR-AT) This dataset includes 1500 turbulence sequences spanning distances from 1 Km to 13 Km. We propose a coarse-to-fine framework to handle severe distortions, which cooperates dynamic turbulence and static background priors.
  arXiv  Detail & Related papers  (2024-07-11T10:38:02Z)
• SFANet: Spatial-Frequency Attention Network for Weather Forecasting [54.470205739015434]
  Weather forecasting plays a critical role in various sectors, driving decision-making and risk management. Traditional methods often struggle to capture the complex dynamics of meteorological systems. We propose a novel framework designed to address these challenges and enhance the accuracy of weather prediction.
  arXiv  Detail & Related papers  (2024-05-29T08:00:15Z)
• CLeaRForecast: Contrastive Learning of High-Purity Representations for Time Series Forecasting [2.5816901096123863]
  Time series forecasting (TSF) holds significant importance in modern society, spanning numerous domains. Previous representation learning-based TSF algorithms typically embrace a contrastive learning paradigm featuring segregated trend-periodicity representations. We propose CLeaRForecast, a novel contrastive learning framework to learn high-purity time series representations with proposed sample, feature, and architecture purifying methods.
  arXiv  Detail & Related papers  (2023-12-10T04:37:43Z)
• Typhoon Intensity Prediction with Vision Transformer [51.84456610977905]
  We introduce "Typhoon Intensity Transformer" (Tint) to predict typhoon intensity accurately across space and time. Tint uses self-attention mechanisms with global receptive fields per layer. Experiments on a publicly available typhoon benchmark validate the efficacy of Tint.
  arXiv  Detail & Related papers  (2023-11-28T03:11:33Z)
• AT-DDPM: Restoring Faces degraded by Atmospheric Turbulence using Denoising Diffusion Probabilistic Models [64.24948495708337]
  Atmospheric turbulence causes significant degradation to image quality by introducing blur and geometric distortion. Various deep learning-based single image atmospheric turbulence mitigation methods, including CNN-based and GAN inversion-based, have been proposed. Denoising Diffusion Probabilistic Models (DDPMs) have recently gained some traction because of their stable training process and their ability to generate high quality images.
  arXiv  Detail & Related papers  (2022-08-24T03:13:04Z)
• Single Frame Atmospheric Turbulence Mitigation: A Benchmark Study and A New Physics-Inspired Transformer Model [82.23276183684001]
  We propose a physics-inspired transformer model for imaging through atmospheric turbulence. The proposed network utilizes the power of transformer blocks to jointly extract a dynamical turbulence distortion map. We present two new real-world turbulence datasets that allow for evaluation with both classical objective metrics and a new task-driven metric using text recognition accuracy.
  arXiv  Detail & Related papers  (2022-07-20T17:09:16Z)
• Imaging through the Atmosphere using Turbulence Mitigation Transformer [15.56320865332645]
  Restoring images distorted by atmospheric turbulence is a ubiquitous problem in long-range imaging applications. Existing deep-learning-based methods have demonstrated promising results in specific testing conditions. We introduce the turbulence mitigation transformer (TMT) that explicitly addresses these issues.
  arXiv  Detail & Related papers  (2022-07-13T18:33:26Z)
• Atmospheric Turbulence Removal with Complex-Valued Convolutional Neural Network [2.657505380055164]
  Atmospheric turbulence distorts visual imagery and is always problematic for information interpretation by both human and machine. Deep learning-based approaches have gained more attention but currently work efficiently only on static scenes. This paper presents a novel learning-based framework offering short temporal spanning to support dynamic scenes.
  arXiv  Detail & Related papers  (2022-04-14T14:29:32Z)
• T$^2$-Net: A Semi-supervised Deep Model for Turbulence Forecasting [65.498967509424]
  Air turbulence forecasting can help airlines avoid hazardous turbulence, guide routes that keep passengers safe, maximize efficiency, reduce costs. Traditional forecasting approaches rely on painstakingly customized turbulence indexes, which are less effective in dynamic and complex weather conditions. We propose a machine learning based turbulence forecasting system due to two challenges: (1) Complex-temporal correlations, and (2) scarcity, very limited turbulence labels can be obtained.
  arXiv  Detail & Related papers  (2020-10-26T21:14:15Z)
• Learn to cycle: Time-consistent feature discovery for action recognition [83.43682368129072]
  Generalizing over temporal variations is a prerequisite for effective action recognition in videos. We introduce Squeeze Re Temporal Gates (SRTG), an approach that favors temporal activations with potential variations. We show consistent improvement when using SRTPG blocks, with only a minimal increase in the number of GFLOs.
  arXiv  Detail & Related papers  (2020-06-15T09:36:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.