Global 3D Reconstruction of Clouds & Tropical Cyclones

• URL: http://arxiv.org/abs/2511.04773v1
• Date: Thu, 06 Nov 2025 19:47:03 GMT
• Title: Global 3D Reconstruction of Clouds & Tropical Cyclones
• Abstract summary: We introduce a new framework to translate 2D satellite imagery into 3D cloud maps of relevant cloud properties.<n>We show that we can - for the first time - create global instantaneous 3D cloud maps and accurately reconstruct the 3D structure of intense storms.
• Score: 0.5523946315455222
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Accurate forecasting of tropical cyclones (TCs) remains challenging due to limited satellite observations probing TC structure and difficulties in resolving cloud properties involved in TC intensification. Recent research has demonstrated the capabilities of machine learning methods for 3D cloud reconstruction from satellite observations. However, existing approaches have been restricted to regions where TCs are uncommon, and are poorly validated for intense storms. We introduce a new framework, based on a pre-training--fine-tuning pipeline, that learns from multiple satellites with global coverage to translate 2D satellite imagery into 3D cloud maps of relevant cloud properties. We apply our model to a custom-built TC dataset to evaluate performance in the most challenging and relevant conditions. We show that we can - for the first time - create global instantaneous 3D cloud maps and accurately reconstruct the 3D structure of intense storms. Our model not only extends available satellite observations but also provides estimates when observations are missing entirely. This is crucial for advancing our understanding of TC intensification and improving forecasts.

Related papers

• CERBERUS: A Three-Headed Decoder for Vertical Cloud Profiles [0.0]
  Atmospheric clouds exhibit complex three-dimensional structure and microphysical details.<n>This mismatch complicates data-driven learning and evaluation of cloud processes in weather and climate models.<n>We introduce CERBERUS, a probabilistic inference framework for generating vertical radar reflectivity profiles.
  arXiv  Detail & Related papers  (2026-04-09T21:16:42Z)
• Improving Ensemble Forecasts of Abnormally Deflecting Tropical Cyclones with Fused Atmosphere-Ocean-Terrain Data [13.045445356929479]
  We present two groundbreaking contributions to deep learning-based tropical cyclone forecasting.<n>First, we have constructed a multimodal and multi-source dataset named AOT-TCs for TC forecasting in the Northwest Pacific basin.<n>Second, based on the AOT-TCs dataset, we propose a forecasting model that can handle both normal and abnormally deflected TCs.
  arXiv  Detail & Related papers  (2026-03-31T03:11:46Z)
• Cloud4D: Estimating Cloud Properties at a High Spatial and Temporal Resolution [8.745444691797067]
  We present Cloud4D, the first learning-based framework that reconstructs a physically consistent, four-dimensional cloud state.<n>Our system delivers an order-of-magnitude improvement in space-time resolution relative to state-of-the-art satellite measurements.
  arXiv  Detail & Related papers  (2025-11-24T18:59:37Z)
• TUM2TWIN: Introducing the Large-Scale Multimodal Urban Digital Twin Benchmark Dataset [90.97440987655084]
  Urban Digital Twins (UDTs) have become essential for managing cities and integrating complex, heterogeneous data from diverse sources.<n>To address these challenges, we introduce the first comprehensive multimodal Urban Digital Twin benchmark dataset: TUM2TWIN.<n>This dataset includes georeferenced, semantically aligned 3D models and networks along with various terrestrial, mobile, aerial, and satellite observations boasting 32 data subsets over roughly 100,000 $m2$ and currently 767 GB of data.
  arXiv  Detail & Related papers  (2025-05-12T09:48:32Z)
• Skillful Nowcasting of Convective Clouds With a Cascade Diffusion Model [16.57308701942378]
  We introduce SATcast, a diffusion model that leverages a cascade architecture and multimodal inputs for nowcasting cloud fields in satellite imagery.<n> SATcast outperforms conventional methods on multiple metrics, demonstrating its superior accuracy and robustness.<n> Notably, SATcast maintains predictive skill for up to 24 hours, underscoring its potential for operational nowcasting applications.
  arXiv  Detail & Related papers  (2025-02-16T02:29:13Z)
• 3D Cloud reconstruction through geospatially-aware Masked Autoencoders [1.4124182346539256]
  This study leverages geostationary imagery from MSG/SEVIRI and radar reflectivity measurements of cloud profiles from CloudSat/CPR to reconstruct 3D cloud structures.<n>We first apply self-supervised learning (SSL) methods-Masked Autoencoders (MAE) and geospatially-aware SatMAE on unlabelled MSG images, and then fine-tune our models on matched image-profile pairs.
  arXiv  Detail & Related papers  (2025-01-03T12:26:04Z)
• Deep Learning for Sea Surface Temperature Reconstruction under Cloud Occlusion [34.00878406145686]
  We describe several Machine Learning models to fill the cloud-occluded areas starting from MODIS Aqua nighttime L3 images.<n>To tackle this challenge, we employed a type of Convolutional Neural Network model (U-net) to reconstruct cloud-covered portions of satellite imagery.<n>Our best-performing architecture show 50% lower root mean square errors over established gap-filling methods.
  arXiv  Detail & Related papers  (2024-12-04T15:49:49Z)
• MambaDS: Near-Surface Meteorological Field Downscaling with Topography Constrained Selective State Space Modeling [68.69647625472464]
  Downscaling, a crucial task in meteorological forecasting, enables the reconstruction of high-resolution meteorological states for target regions. Previous downscaling methods lacked tailored designs for meteorology and encountered structural limitations. We propose a novel model called MambaDS, which enhances the utilization of multivariable correlations and topography information.
  arXiv  Detail & Related papers  (2024-08-20T13:45:49Z)
• Learning Robust Precipitation Forecaster by Temporal Frame Interpolation [65.5045412005064]
  We develop a robust precipitation forecasting model that demonstrates resilience against spatial-temporal discrepancies. Our approach has led to significant improvements in forecasting precision, culminating in our model securing textit1st place in the transfer learning leaderboard of the textitWeather4cast'23 competition.
  arXiv  Detail & Related papers  (2023-11-30T08:22:08Z)
• Transformer-based nowcasting of radar composites from satellite images for severe weather [45.0983299269404]
  We present a Transformer-based model for nowcasting ground-based radar image sequences using satellite data up to two hours lead time. Trained on a dataset reflecting severe weather conditions, the model predicts radar fields occurring under different weather phenomena. The model can support precipitation nowcasting across large domains without an explicit need for radar towers.
  arXiv  Detail & Related papers  (2023-10-30T13:17:38Z)
• SST: Real-time End-to-end Monocular 3D Reconstruction via Sparse Spatial-Temporal Guidance [71.3027345302485]
  Real-time monocular 3D reconstruction is a challenging problem that remains unsolved. We propose an end-to-end 3D reconstruction network SST, which utilizes Sparse estimated points from visual SLAM system. SST outperforms all state-of-the-art competitors, whilst keeping a high inference speed at 59 FPS.
  arXiv  Detail & Related papers  (2022-12-13T12:17:13Z)
• Deep Point Cloud Reconstruction [74.694733918351]
  Point cloud obtained from 3D scanning is often sparse, noisy, and irregular. To cope with these issues, recent studies have been separately conducted to densify, denoise, and complete inaccurate point cloud. We propose a deep point cloud reconstruction network consisting of two stages: 1) a 3D sparse stacked-hourglass network as for the initial densification and denoising, 2) a refinement via transformers converting the discrete voxels into 3D points.
  arXiv  Detail & Related papers  (2021-11-23T07:53:28Z)
• Pseudo-LiDAR Point Cloud Interpolation Based on 3D Motion Representation and Spatial Supervision [68.35777836993212]
  We propose a Pseudo-LiDAR point cloud network to generate temporally and spatially high-quality point cloud sequences. By exploiting the scene flow between point clouds, the proposed network is able to learn a more accurate representation of the 3D spatial motion relationship.
  arXiv  Detail & Related papers  (2020-06-20T03:11:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.