A Comparative Study on Generative Models for High Resolution Solar Observation Imaging

• URL: http://arxiv.org/abs/2304.07169v1
• Date: Fri, 14 Apr 2023 14:40:32 GMT
• Title: A Comparative Study on Generative Models for High Resolution Solar Observation Imaging
• Authors: Mehdi Cherti, Alexander Czernik, Stefan Kesselheim, Frederic Effenberger, Jenia Jitsev
• Abstract summary: This work investigates capabilities of current state-of-the-art generative models to accurately capture the data distribution behind observed solar activity states. Using distributed training on supercomputers, we are able to train generative models for up to 1024x1024 resolution that produce high quality samples indistinguishable to human experts.
• Score: 59.372588316558826
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Solar activity is one of the main drivers of variability in our solar system and the key source of space weather phenomena that affect Earth and near Earth space. The extensive record of high resolution extreme ultraviolet (EUV) observations from the Solar Dynamics Observatory (SDO) offers an unprecedented, very large dataset of solar images. In this work, we make use of this comprehensive dataset to investigate capabilities of current state-of-the-art generative models to accurately capture the data distribution behind the observed solar activity states. Starting from StyleGAN-based methods, we uncover severe deficits of this model family in handling fine-scale details of solar images when training on high resolution samples, contrary to training on natural face images. When switching to the diffusion based generative model family, we observe strong improvements of fine-scale detail generation. For the GAN family, we are able to achieve similar improvements in fine-scale generation when turning to ProjectedGANs, which uses multi-scale discriminators with a pre-trained frozen feature extractor. We conduct ablation studies to clarify mechanisms responsible for proper fine-scale handling. Using distributed training on supercomputers, we are able to train generative models for up to 1024x1024 resolution that produce high quality samples indistinguishable to human experts, as suggested by the evaluation we conduct. We make all code, models and workflows used in this study publicly available at \url{https://github.com/SLAMPAI/generative-models-for-highres-solar-images}.

Related papers

• Solar synthetic imaging: Introducing denoising diffusion probabilistic models on SDO/AIA data [0.0]
  This study proposes using generative deep learning models, specifically a Denoising Diffusion Probabilistic Model (DDPM), to create synthetic images of solar phenomena. By employing a dataset from the AIA instrument aboard the SDO spacecraft, we aim to address the data scarcity issue. The DDPM's performance is evaluated using cluster metrics, Frechet Inception Distance (FID), and F1-score, showcasing promising results in generating realistic solar imagery.
  arXiv  Detail & Related papers  (2024-04-03T08:18:45Z)
• Self-Play Fine-Tuning of Diffusion Models for Text-to-Image Generation [59.184980778643464]
  Fine-tuning Diffusion Models remains an underexplored frontier in generative artificial intelligence (GenAI) In this paper, we introduce an innovative technique called self-play fine-tuning for diffusion models (SPIN-Diffusion) Our approach offers an alternative to conventional supervised fine-tuning and RL strategies, significantly improving both model performance and alignment.
  arXiv  Detail & Related papers  (2024-02-15T18:59:18Z)
• High-Cadence Thermospheric Density Estimation enabled by Machine Learning on Solar Imagery [0.14061979259370275]
  We incorporate NASA's Solar Dynamics Observatory (SDO) extreme ultraviolet (EUV) spectral images into a neural thermospheric density model. We demonstrate that EUV imagery can enable predictions with much higher temporal resolution and replace ground-based proxies.
  arXiv  Detail & Related papers  (2023-11-12T23:39:21Z)
• Sky-image-based solar forecasting using deep learning with multi-location data: training models locally, globally or via transfer learning? [0.0]
  One of the biggest challenges for training deep learning models is the availability of labeled datasets. With more and more sky image datasets open sourced in recent years, the development of accurate and reliable solar forecasting methods has seen a huge growth in potential.
  arXiv  Detail & Related papers  (2022-11-03T19:25:28Z)
• Learning Generative Vision Transformer with Energy-Based Latent Space for Saliency Prediction [51.80191416661064]
  We propose a novel vision transformer with latent variables following an informative energy-based prior for salient object detection. Both the vision transformer network and the energy-based prior model are jointly trained via Markov chain Monte Carlo-based maximum likelihood estimation. With the generative vision transformer, we can easily obtain a pixel-wise uncertainty map from an image, which indicates the model confidence in predicting saliency from the image.
  arXiv  Detail & Related papers  (2021-12-27T06:04:33Z)
• InvGAN: Invertible GANs [88.58338626299837]
  InvGAN, short for Invertible GAN, successfully embeds real images to the latent space of a high quality generative model. This allows us to perform image inpainting, merging, and online data augmentation.
  arXiv  Detail & Related papers  (2021-12-08T21:39:00Z)
• Toward Foundation Models for Earth Monitoring: Proposal for a Climate Change Benchmark [95.19070157520633]
  Recent progress in self-supervision shows that pre-training large neural networks on vast amounts of unsupervised data can lead to impressive increases in generalisation for downstream tasks. Such models, recently coined as foundation models, have been transformational to the field of natural language processing. We propose to develop a new benchmark comprised of a variety of downstream tasks related to climate change.
  arXiv  Detail & Related papers  (2021-12-01T15:38:19Z)
• Benchmarking of Deep Learning Irradiance Forecasting Models from Sky Images -- an in-depth Analysis [0.0]
  We train four commonly used Deep Learning architectures to forecast solar irradiance from sequences of hemispherical sky images. Results show that encodingtemporal aspects greatly improved the predictions with 10 min Forecast Skill reaching 20.4% on the test year. We conclude that, with a common setup, Deep Learning models tend to behave just as a'very smart persistence model', temporally aligned with the persistence model while mitigating its most penalising errors.
  arXiv  Detail & Related papers  (2021-02-01T09:31:14Z)
• Estimating Crop Primary Productivity with Sentinel-2 and Landsat 8 using Machine Learning Methods Trained with Radiative Transfer Simulations [58.17039841385472]
  We take advantage of all parallel developments in mechanistic modeling and satellite data availability for advanced monitoring of crop productivity. Our model successfully estimates gross primary productivity across a variety of C3 crop types and environmental conditions even though it does not use any local information from the corresponding sites. This highlights its potential to map crop productivity from new satellite sensors at a global scale with the help of current Earth observation cloud computing platforms.
  arXiv  Detail & Related papers  (2020-12-07T16:23:13Z)
• Predicting Landsat Reflectance with Deep Generative Fusion [2.867517731896504]
  Public satellite missions are commonly bound to a trade-off between spatial and temporal resolution. This hinders their potential to assist vegetation monitoring or humanitarian actions. We probe the potential of deep generative models to produce high-resolution optical imagery by fusing products with different spatial and temporal characteristics.
  arXiv  Detail & Related papers  (2020-11-09T21:06:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.