TurbuGAN: An Adversarial Learning Approach to Spatially-Varying Multiframe Blind Deconvolution with Applications to Imaging Through Turbulence

• URL: http://arxiv.org/abs/2203.06764v1
• Date: Sun, 13 Mar 2022 21:32:34 GMT
• Title: TurbuGAN: An Adversarial Learning Approach to Spatially-Varying Multiframe Blind Deconvolution with Applications to Imaging Through Turbulence
• Authors: Brandon Y. Feng, Mingyang Xie, Christopher A. Metzler
• Abstract summary: We present a self-supervised and self-calibrating multi-shot approach to imaging through atmospheric turbulence, called TurbuGAN. Our approach requires no paired training data, adapts itself to the distribution of the turbulence, leverages domain-specific data priors, outperforms existing approaches, and can generalize from tens to tens of thousands of measurements.
• Score: 9.156939957189504
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a self-supervised and self-calibrating multi-shot approach to imaging through atmospheric turbulence, called TurbuGAN. Our approach requires no paired training data, adapts itself to the distribution of the turbulence, leverages domain-specific data priors, outperforms existing approaches, and can generalize from tens to tens of thousands of measurements. We achieve such functionality through an adversarial sensing framework adapted from CryoGAN, which uses a discriminator network to match the distributions of captured and simulated measurements. Our framework builds on CryoGAN by (1) generalizing the forward measurement model to incorporate physically accurate and computationally efficient models for light propagation through anisoplanatic turbulence, (2) enabling adaptation to slightly misspecified forward models, and (3) leveraging domain-specific prior knowledge using pretrained generative networks, when available. We validate TurbuGAN in simulation using realistic models for atmospheric turbulence-induced distortion.

Related papers

• Koopman-Based Surrogate Modelling of Turbulent Rayleigh-Bénard Convection [4.248022697109535]
  We use a Koopman-inspired architecture called the Linear Recurrent Autoencoder Network (LRAN) for learning reduced-order dynamics in convection flows. A traditional fluid dynamics method, the Kernel Dynamic Mode Decomposition (KDMD) is used to compare the LRAN. We obtained more accurate predictions with the LRAN than with KDMD in the most turbulent setting.
  arXiv  Detail & Related papers  (2024-05-10T12:15:02Z)
• Guided Flows for Generative Modeling and Decision Making [55.42634941614435]
  We show that Guided Flows significantly improves the sample quality in conditional image generation and zero-shot text synthesis-to-speech. Notably, we are first to apply flow models for plan generation in the offline reinforcement learning setting ax speedup in compared to diffusion models.
  arXiv  Detail & Related papers  (2023-11-22T15:07:59Z)
• Bayesian Conditional Diffusion Models for Versatile Spatiotemporal Turbulence Generation [13.278744447861289]
  We introduce a novel generative framework grounded in probabilistic diffusion models for turbulence generation. A notable feature of our approach is the proposed method for long-span flow sequence generation, which is based on autoregressive-based conditional sampling. We showcase the versatile turbulence generation capability of our framework through a suite of numerical experiments.
  arXiv  Detail & Related papers  (2023-11-14T04:08:14Z)
• Over-the-Air Federated Learning and Optimization [52.5188988624998]
  We focus on Federated learning (FL) via edge-the-air computation (AirComp) We describe the convergence of AirComp-based FedAvg (AirFedAvg) algorithms under both convex and non- convex settings. For different types of local updates that can be transmitted by edge devices (i.e., model, gradient, model difference), we reveal that transmitting in AirFedAvg may cause an aggregation error. In addition, we consider more practical signal processing schemes to improve the communication efficiency and extend the convergence analysis to different forms of model aggregation error caused by these signal processing schemes.
  arXiv  Detail & Related papers  (2023-10-16T05:49:28Z)
• Inferring Traffic Models in Terminal Airspace from Flight Tracks and Procedures [52.25258289718559]
  We propose a probabilistic model that can learn the variability from procedural data and flight tracks collected from radar surveillance data. We show how a pairwise model can be used to generate traffic involving an arbitrary number of aircraft.
  arXiv  Detail & Related papers  (2023-03-17T13:58:06Z)
• Single Frame Atmospheric Turbulence Mitigation: A Benchmark Study and A New Physics-Inspired Transformer Model [82.23276183684001]
  We propose a physics-inspired transformer model for imaging through atmospheric turbulence. The proposed network utilizes the power of transformer blocks to jointly extract a dynamical turbulence distortion map. We present two new real-world turbulence datasets that allow for evaluation with both classical objective metrics and a new task-driven metric using text recognition accuracy.
  arXiv  Detail & Related papers  (2022-07-20T17:09:16Z)
• Dream to Explore: Adaptive Simulations for Autonomous Systems [3.0664963196464448]
  We tackle the problem of learning to control dynamical systems by applying Bayesian nonparametric methods. By employing Gaussian processes to discover latent world dynamics, we mitigate common data efficiency issues observed in reinforcement learning. Our algorithm jointly learns a world model and policy by optimizing a variational lower bound of a log-likelihood.
  arXiv  Detail & Related papers  (2021-10-27T04:27:28Z)
• Data-Driven Wind Turbine Wake Modeling via Probabilistic Machine Learning [0.0]
  We use real-world light detection and ranging (LiDAR) measurements of wind-turbine wakes to construct predictive surrogate models using machine learning. We find that our approach provides accurate approximations of the wind-turbine wake flow field that can be queried at an orders-of-magnitude cheaper cost than those generated with high-fidelity physics-based simulations.
  arXiv  Detail & Related papers  (2021-09-06T14:46:20Z)
• DAIS: Automatic Channel Pruning via Differentiable Annealing Indicator Search [55.164053971213576]
  convolutional neural network has achieved great success in fulfilling computer vision tasks despite large computation overhead. Structured (channel) pruning is usually applied to reduce the model redundancy while preserving the network structure. Existing structured pruning methods require hand-crafted rules which may lead to tremendous pruning space.
  arXiv  Detail & Related papers  (2020-11-04T07:43:01Z)
• T$^2$-Net: A Semi-supervised Deep Model for Turbulence Forecasting [65.498967509424]
  Air turbulence forecasting can help airlines avoid hazardous turbulence, guide routes that keep passengers safe, maximize efficiency, reduce costs. Traditional forecasting approaches rely on painstakingly customized turbulence indexes, which are less effective in dynamic and complex weather conditions. We propose a machine learning based turbulence forecasting system due to two challenges: (1) Complex-temporal correlations, and (2) scarcity, very limited turbulence labels can be obtained.
  arXiv  Detail & Related papers  (2020-10-26T21:14:15Z)
• Automating Turbulence Modeling by Multi-Agent Reinforcement Learning [4.784658158364452]
  We introduce multi-agent reinforcement learning as an automated discovery tool of turbulence models. We demonstrate the potential of this approach on Large Eddy Simulations of homogeneous and isotropic turbulence.
  arXiv  Detail & Related papers  (2020-05-18T18:45:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.