Spatio-Temporal Turbulence Mitigation: A Translational Perspective

• URL: http://arxiv.org/abs/2401.04244v2
• Date: Sun, 7 Apr 2024 20:13:45 GMT
• Title: Spatio-Temporal Turbulence Mitigation: A Translational Perspective
• Authors: Xingguang Zhang, Nicholas Chimitt, Yiheng Chi, Zhiyuan Mao, Stanley H. Chan,
• Abstract summary: We present the Deep Atmospheric TUrbulence Mitigation network ( DATUM) DATUM aims to overcome major challenges when transitioning from classical to deep learning approaches. A large-scale training dataset, ATSyn, is presented as a co-invention to enable generalization in real turbulence.
• Score: 13.978156774471744
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recovering images distorted by atmospheric turbulence is a challenging inverse problem due to the stochastic nature of turbulence. Although numerous turbulence mitigation (TM) algorithms have been proposed, their efficiency and generalization to real-world dynamic scenarios remain severely limited. Building upon the intuitions of classical TM algorithms, we present the Deep Atmospheric TUrbulence Mitigation network (DATUM). DATUM aims to overcome major challenges when transitioning from classical to deep learning approaches. By carefully integrating the merits of classical multi-frame TM methods into a deep network structure, we demonstrate that DATUM can efficiently perform long-range temporal aggregation using a recurrent fashion, while deformable attention and temporal-channel attention seamlessly facilitate pixel registration and lucky imaging. With additional supervision, tilt and blur degradation can be jointly mitigated. These inductive biases empower DATUM to significantly outperform existing methods while delivering a tenfold increase in processing speed. A large-scale training dataset, ATSyn, is presented as a co-invention to enable generalization in real turbulence. Our code and datasets are available at https://xg416.github.io/DATUM.

Related papers

• Continuous Exposure-Time Modeling for Realistic Atmospheric Turbulence Synthesis [65.19146708498346]
  Atmospheric turbulence significantly degrades long-range imaging by introducing geometric warping and exposure-timedependent blur.<n>Existing methods for turbulence effects often oversimplify the relationship between blur and exposure-time.<n>We construct ET-Turb, a large-scale synthetic turbulence dataset that explicitly incorporates continuous exposure-time modeling.
  arXiv  Detail & Related papers  (2026-03-02T02:58:44Z)
• MEMTS: Internalizing Domain Knowledge via Parameterized Memory for Retrieval-Free Domain Adaptation of Time Series Foundation Models [51.506429027626005]
  Memory for Time Series (MEMTS) is a lightweight and plug-and-play method for retrieval-free domain adaptation in time series forecasting.<n>Key component of MEMTS is a Knowledge Persistence Module (KPM), which internalizes domain-specific temporal dynamics.<n>This paradigm shift enables MEMTS to achieve accurate domain adaptation with constant-time inference and near-zero latency.
  arXiv  Detail & Related papers  (2026-02-14T14:00:06Z)
• RainDiff: End-to-end Precipitation Nowcasting Via Token-wise Attention Diffusion [64.49056527678606]
  We propose a Token-wise Attention integrated into not only the U-Net diffusion model but also the radar-temporal encoder.<n>Unlike prior approaches, our method integrates attention into the architecture without incurring the high resource cost typical of pixel-space diffusion.<n>Our experiments and evaluations demonstrate that the proposed method significantly outperforms state-of-the-art approaches, robustness local fidelity, generalization, and superior in complex precipitation forecasting scenarios.
  arXiv  Detail & Related papers  (2025-10-16T17:59:13Z)
• EGTM: Event-guided Efficient Turbulence Mitigation [19.09752432962073]
  Turbulence mitigation (TM) aims to remove the distortions and blurs introduced by atmospheric turbulence into frame cameras.<n>We present a novel EGTM framework that extracts pixel-level reliable turbulence-free guidance from noisy turbulent events for temporal lucky fusion.<n>We build the first turbulence data acquisition system to contribute the first real-world event-driven TM dataset.
  arXiv  Detail & Related papers  (2025-09-04T01:49:13Z)
• RMFAT: Recurrent Multi-scale Feature Atmospheric Turbulence Mitigator [4.021926055330021]
  Atmospheric turbulence severely degrades video quality by introducing distortions such as geometric warping, blur, and temporal flickering.<n>We propose RMFAT: Recurrent Multi-scale Feature Atmospheric Turbulence Mitigator, designed for efficient and temporally consistent video restoration under AT conditions.
  arXiv  Detail & Related papers  (2025-08-15T11:20:18Z)
• StPR: Spatiotemporal Preservation and Routing for Exemplar-Free Video Class-Incremental Learning [51.003833566279006]
  Class-Incremental Learning (CIL) seeks to develop models that continuously learn new action categories over time without previously acquired knowledge.<n>Existing approaches either rely on forgetting, raising concerns over memory and privacy, or adapt static image-based methods that neglect temporal modeling.<n>We propose a unified and exemplar-free VCIL framework that explicitly disentangles and preserves information.
  arXiv  Detail & Related papers  (2025-05-20T06:46:51Z)
• Learning Phase Distortion with Selective State Space Models for Video Turbulence Mitigation [13.073844945948132]
  Atmospheric turbulence is a major source of image degradation in long-range imaging systems. Many deep learning-based turbulence mitigation (TM) methods have been proposed, but they are slow, memory-hungry, and do not generalize well. We present a new TM method based on two concepts: (1) A turbulence mitigation network based on the Selective State Space Model (MambaTM) and (2) Learned Latent Phase Distortion (LPD) Our proposed method exceeds current state-of-the-art networks on various synthetic and real-world TM benchmarks with significantly faster inference speed.
  arXiv  Detail & Related papers  (2025-04-03T15:33:18Z)
• Long-range Turbulence Mitigation: A Large-scale Dataset and A Coarse-to-fine Framework [12.869617522032986]
  We construct a large-scale real long-range atmospheric turbulence dataset (RLR-AT) This dataset includes 1500 turbulence sequences spanning distances from 1 Km to 13 Km. We propose a coarse-to-fine framework to handle severe distortions, which cooperates dynamic turbulence and static background priors.
  arXiv  Detail & Related papers  (2024-07-11T10:38:02Z)
• SFANet: Spatial-Frequency Attention Network for Weather Forecasting [54.470205739015434]
  Weather forecasting plays a critical role in various sectors, driving decision-making and risk management. Traditional methods often struggle to capture the complex dynamics of meteorological systems. We propose a novel framework designed to address these challenges and enhance the accuracy of weather prediction.
  arXiv  Detail & Related papers  (2024-05-29T08:00:15Z)
• CLeaRForecast: Contrastive Learning of High-Purity Representations for Time Series Forecasting [2.5816901096123863]
  Time series forecasting (TSF) holds significant importance in modern society, spanning numerous domains. Previous representation learning-based TSF algorithms typically embrace a contrastive learning paradigm featuring segregated trend-periodicity representations. We propose CLeaRForecast, a novel contrastive learning framework to learn high-purity time series representations with proposed sample, feature, and architecture purifying methods.
  arXiv  Detail & Related papers  (2023-12-10T04:37:43Z)
• Typhoon Intensity Prediction with Vision Transformer [51.84456610977905]
  We introduce "Typhoon Intensity Transformer" (Tint) to predict typhoon intensity accurately across space and time. Tint uses self-attention mechanisms with global receptive fields per layer. Experiments on a publicly available typhoon benchmark validate the efficacy of Tint.
  arXiv  Detail & Related papers  (2023-11-28T03:11:33Z)
• AT-DDPM: Restoring Faces degraded by Atmospheric Turbulence using Denoising Diffusion Probabilistic Models [64.24948495708337]
  Atmospheric turbulence causes significant degradation to image quality by introducing blur and geometric distortion. Various deep learning-based single image atmospheric turbulence mitigation methods, including CNN-based and GAN inversion-based, have been proposed. Denoising Diffusion Probabilistic Models (DDPMs) have recently gained some traction because of their stable training process and their ability to generate high quality images.
  arXiv  Detail & Related papers  (2022-08-24T03:13:04Z)
• Single Frame Atmospheric Turbulence Mitigation: A Benchmark Study and A New Physics-Inspired Transformer Model [82.23276183684001]
  We propose a physics-inspired transformer model for imaging through atmospheric turbulence. The proposed network utilizes the power of transformer blocks to jointly extract a dynamical turbulence distortion map. We present two new real-world turbulence datasets that allow for evaluation with both classical objective metrics and a new task-driven metric using text recognition accuracy.
  arXiv  Detail & Related papers  (2022-07-20T17:09:16Z)
• Imaging through the Atmosphere using Turbulence Mitigation Transformer [15.56320865332645]
  Restoring images distorted by atmospheric turbulence is a ubiquitous problem in long-range imaging applications. Existing deep-learning-based methods have demonstrated promising results in specific testing conditions. We introduce the turbulence mitigation transformer (TMT) that explicitly addresses these issues.
  arXiv  Detail & Related papers  (2022-07-13T18:33:26Z)
• Atmospheric Turbulence Removal with Complex-Valued Convolutional Neural Network [2.657505380055164]
  Atmospheric turbulence distorts visual imagery and is always problematic for information interpretation by both human and machine. Deep learning-based approaches have gained more attention but currently work efficiently only on static scenes. This paper presents a novel learning-based framework offering short temporal spanning to support dynamic scenes.
  arXiv  Detail & Related papers  (2022-04-14T14:29:32Z)
• T$^2$-Net: A Semi-supervised Deep Model for Turbulence Forecasting [65.498967509424]
  Air turbulence forecasting can help airlines avoid hazardous turbulence, guide routes that keep passengers safe, maximize efficiency, reduce costs. Traditional forecasting approaches rely on painstakingly customized turbulence indexes, which are less effective in dynamic and complex weather conditions. We propose a machine learning based turbulence forecasting system due to two challenges: (1) Complex-temporal correlations, and (2) scarcity, very limited turbulence labels can be obtained.
  arXiv  Detail & Related papers  (2020-10-26T21:14:15Z)
• Learn to cycle: Time-consistent feature discovery for action recognition [83.43682368129072]
  Generalizing over temporal variations is a prerequisite for effective action recognition in videos. We introduce Squeeze Re Temporal Gates (SRTG), an approach that favors temporal activations with potential variations. We show consistent improvement when using SRTPG blocks, with only a minimal increase in the number of GFLOs.
  arXiv  Detail & Related papers  (2020-06-15T09:36:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.