Intermediate and Future Frame Prediction of Geostationary Satellite Imagery With Warp and Refine Network

• URL: http://arxiv.org/abs/2303.04405v1
• Date: Wed, 8 Mar 2023 06:53:42 GMT
• Title: Intermediate and Future Frame Prediction of Geostationary Satellite Imagery With Warp and Refine Network
• Authors: Minseok Seo, Yeji Choi, Hyungon Ry, Heesun Park, Hyungkun Bae, Hyesook Lee, Wanseok Seo
• Abstract summary: Geostationary satellite imagery has applications in climate and weather forecasting, planning natural energy resources, and predicting extreme weather events. For precise and accurate prediction, higher spatial and temporal resolution of geostationary satellite imagery is important. We proposed warp and refine network (WR-Net) to solve this problem. WR-Net is divided into an optical flow warp component and a warp image refinement component.
• Score: 4.393292642453663
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Geostationary satellite imagery has applications in climate and weather forecasting, planning natural energy resources, and predicting extreme weather events. For precise and accurate prediction, higher spatial and temporal resolution of geostationary satellite imagery is important. Although recent geostationary satellite resolution has improved, the long-term analysis of climate applications is limited to using multiple satellites from the past to the present due to the different resolutions. To solve this problem, we proposed warp and refine network (WR-Net). WR-Net is divided into an optical flow warp component and a warp image refinement component. We used the TV-L1 algorithm instead of deep learning-based approaches to extract the optical flow warp component. The deep-learning-based model is trained on the human-centric view of the RGB channel and does not work on geostationary satellites, which is gray-scale one-channel imagery. The refinement network refines the warped image through a multi-temporal fusion layer. We evaluated WR-Net by interpolation of temporal resolution at 4 min intervals to 2 min intervals in large-scale GK2A geostationary meteorological satellite imagery. Furthermore, we applied WR-Net to the future frame prediction task and showed that the explicit use of optical flow can help future frame prediction.

Related papers

• DiffusionSat: A Generative Foundation Model for Satellite Imagery [63.2807119794691]
  We present DiffusionSat, to date the largest generative foundation model trained on a collection of publicly available large, high-resolution remote sensing datasets. Our method produces realistic samples and can be used to solve multiple generative tasks including temporal generation, superresolution given multi-spectral inputs and in-painting.
  arXiv  Detail & Related papers  (2023-12-06T16:53:17Z)
• WeatherFusionNet: Predicting Precipitation from Satellite Data [0.0]
  We aim to predict high-resolution precipitation from lower-resolution satellite radiance images. A neural network called WeatherFusionNet is employed to predict severe rain up to eight hours in advance. We achieved 1st place in the NeurIPS 2022 Weather4Cast Core challenge.
  arXiv  Detail & Related papers  (2022-11-30T08:49:13Z)
• Meteorological Satellite Images Prediction Based on Deep Multi-scales Extrapolation Fusion [5.125401106179782]
  It is important to make accurate predictions for meteorological satellite images, especially the nowcasting prediction up to 2 hours ahead. Here we present a deep multiscales fusion method, to address the challenge of the nowcasting prediction. Experiments based on the FY-4A meteorological satellite data show that the proposed method can generate realistic prediction images.
  arXiv  Detail & Related papers  (2022-09-19T16:00:17Z)
• Convolutional Neural Processes for Inpainting Satellite Images [56.032183666893246]
  Inpainting involves predicting what is missing based on the known pixels and is an old problem in image processing. We show ConvvNPs can outperform classical methods and state-of-the-art deep learning inpainting models on a scanline inpainting problem for LANDSAT 7 satellite images.
  arXiv  Detail & Related papers  (2022-05-24T23:29:04Z)
• A Deep Learning Ensemble Framework for Off-Nadir Geocentric Pose Prediction [0.0]
  Current software functions optimally only on near-nadir images, though off-nadir images are often the first sources of information following a natural disaster. This study proposes a deep learning ensemble framework to predict geocentric pose using 5,923 near-nadir and off-nadir RGB satellite images of cities worldwide.
  arXiv  Detail & Related papers  (2022-05-04T08:33:41Z)
• Beyond Cross-view Image Retrieval: Highly Accurate Vehicle Localization Using Satellite Image [91.29546868637911]
  This paper addresses the problem of vehicle-mounted camera localization by matching a ground-level image with an overhead-view satellite map. The key idea is to formulate the task as pose estimation and solve it by neural-net based optimization. Experiments on standard autonomous vehicle localization datasets have confirmed the superiority of the proposed method.
  arXiv  Detail & Related papers  (2022-04-10T19:16:58Z)
• Deep Learning for Real Time Satellite Pose Estimation on Low Power Edge TPU [58.720142291102135]
  In this paper we propose a pose estimation software exploiting neural network architectures. We show how low power machine learning accelerators could enable Artificial Intelligence exploitation in space.
  arXiv  Detail & Related papers  (2022-04-07T08:53:18Z)
• Real-time Ionospheric Imaging of S4 Scintillation from Limited Data with Parallel Kalman Filters and Smoothness [91.3755431537592]
  We create two dimensional ionospheric images of S4 amplitude scintillation at 350 km over South America with temporal resolution of one minute. Our results show that in areas with a network of ground receivers with a relatively good coverage the produced images can provide reliable real-time results.
  arXiv  Detail & Related papers  (2021-05-11T23:09:14Z)
• EarthNet2021: A large-scale dataset and challenge for Earth surface forecasting as a guided video prediction task [12.795776149170978]
  We frame Earth surface forecasting as the task of predicting satellite imagery conditioned on future weather. EarthNet2021 is a large dataset suitable for training deep neural networks on the task. Resulting forecasts will greatly improve over the spatial resolution found in numerical models.
  arXiv  Detail & Related papers  (2021-04-16T09:47:30Z)
• Physically-Consistent Generative Adversarial Networks for Coastal Flood Visualization [60.690929022840685]
  We propose the first deep learning pipeline to ensure physical-consistency in synthetic visual satellite imagery. By evaluating the imagery relative to physics-based flood maps, we find that our proposed framework outperforms baseline models in both physical-consistency and photorealism. We publish a dataset of over 25k labelled image-pairs to study image-to-image translation in Earth observation.
  arXiv  Detail & Related papers  (2021-04-10T15:00:15Z)
• H2O-Net: Self-Supervised Flood Segmentation via Adversarial Domain Adaptation and Label Refinement [6.577064131678387]
  This work presents H2O-Network, a self supervised deep learning method to segment floods from satellites and aerial imagery. H2O-Network learns to synthesize signals highly correlative with water presence as a domain adaptation step for semantic segmentation in high resolution satellite imagery. We demonstrate that H2O-Net outperforms the state-of-the-art semantic segmentation methods on satellite imagery by 10% and 12% pixel accuracy and mIoU respectively.
  arXiv  Detail & Related papers  (2020-10-11T18:35:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.