Single Frame Atmospheric Turbulence Mitigation: A Benchmark Study and A
New Physics-Inspired Transformer Model
- URL: http://arxiv.org/abs/2207.10040v1
- Date: Wed, 20 Jul 2022 17:09:16 GMT
- Title: Single Frame Atmospheric Turbulence Mitigation: A Benchmark Study and A
New Physics-Inspired Transformer Model
- Authors: Zhiyuan Mao and Ajay Jaiswal and Zhangyang Wang and Stanley H. Chan
- Abstract summary: We propose a physics-inspired transformer model for imaging through atmospheric turbulence.
The proposed network utilizes the power of transformer blocks to jointly extract a dynamical turbulence distortion map.
We present two new real-world turbulence datasets that allow for evaluation with both classical objective metrics and a new task-driven metric using text recognition accuracy.
- Score: 82.23276183684001
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Image restoration algorithms for atmospheric turbulence are known to be much
more challenging to design than traditional ones such as blur or noise because
the distortion caused by the turbulence is an entanglement of spatially varying
blur, geometric distortion, and sensor noise. Existing CNN-based restoration
methods built upon convolutional kernels with static weights are insufficient
to handle the spatially dynamical atmospheric turbulence effect. To address
this problem, in this paper, we propose a physics-inspired transformer model
for imaging through atmospheric turbulence. The proposed network utilizes the
power of transformer blocks to jointly extract a dynamical turbulence
distortion map and restore a turbulence-free image. In addition, recognizing
the lack of a comprehensive dataset, we collect and present two new real-world
turbulence datasets that allow for evaluation with both classical objective
metrics (e.g., PSNR and SSIM) and a new task-driven metric using text
recognition accuracy. Both real testing sets and all related code will be made
publicly available.
Related papers
- Turb-Seg-Res: A Segment-then-Restore Pipeline for Dynamic Videos with Atmospheric Turbulence [10.8380383565446]
This paper presents the first segment-then-restore pipeline for restoring the videos of dynamic scenes in turbulent environment.
We leverage mean optical flow with an unsupervised motion segmentation method to separate dynamic and static scene components prior to restoration.
Benchmarked against existing restoration methods, our approach restores most of the geometric distortion and enhances sharpness for videos.
arXiv Detail & Related papers (2024-04-21T10:28:34Z) - Physics-Driven Turbulence Image Restoration with Stochastic Refinement [80.79900297089176]
Image distortion by atmospheric turbulence is a critical problem in long-range optical imaging systems.
Fast and physics-grounded simulation tools have been introduced to help the deep-learning models adapt to real-world turbulence conditions.
This paper proposes the Physics-integrated Restoration Network (PiRN) to help the network to disentangle theity from the degradation and the underlying image.
arXiv Detail & Related papers (2023-07-20T05:49:21Z) - AT-DDPM: Restoring Faces degraded by Atmospheric Turbulence using
Denoising Diffusion Probabilistic Models [64.24948495708337]
Atmospheric turbulence causes significant degradation to image quality by introducing blur and geometric distortion.
Various deep learning-based single image atmospheric turbulence mitigation methods, including CNN-based and GAN inversion-based, have been proposed.
Denoising Diffusion Probabilistic Models (DDPMs) have recently gained some traction because of their stable training process and their ability to generate high quality images.
arXiv Detail & Related papers (2022-08-24T03:13:04Z) - Learning to restore images degraded by atmospheric turbulence using
uncertainty [93.72048616001064]
Atmospheric turbulence can significantly degrade the quality of images acquired by long-range imaging systems.
We propose a deep learning-based approach for restring a single image degraded by atmospheric turbulence.
arXiv Detail & Related papers (2022-07-07T17:24:52Z) - A comparison of different atmospheric turbulence simulation methods for
image restoration [64.24948495708337]
Atmospheric turbulence deteriorates the quality of images captured by long-range imaging systems.
Various deep learning-based atmospheric turbulence mitigation methods have been proposed in the literature.
We systematically evaluate the effectiveness of various turbulence simulation methods on image restoration.
arXiv Detail & Related papers (2022-04-19T16:21:36Z) - Atmospheric Turbulence Removal with Complex-Valued Convolutional Neural
Network [2.657505380055164]
Atmospheric turbulence distorts visual imagery and is always problematic for information interpretation by both human and machine.
Deep learning-based approaches have gained more attention but currently work efficiently only on static scenes.
This paper presents a novel learning-based framework offering short temporal spanning to support dynamic scenes.
arXiv Detail & Related papers (2022-04-14T14:29:32Z) - Image Reconstruction of Static and Dynamic Scenes through Anisoplanatic
Turbulence [1.6114012813668934]
We present a unified method for atmospheric turbulence mitigation in both static and dynamic sequences.
We are able to achieve better results compared to existing methods by utilizing a novel space-time non-local averaging method.
arXiv Detail & Related papers (2020-08-31T19:20:46Z) - Learning to Restore a Single Face Image Degraded by Atmospheric
Turbulence using CNNs [93.72048616001064]
Images captured under such condition suffer from a combination of geometric deformation and space varying blur.
We present a deep learning-based solution to the problem of restoring a turbulence-degraded face image.
arXiv Detail & Related papers (2020-07-16T15:25:08Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.