Cloud4D: Estimating Cloud Properties at a High Spatial and Temporal Resolution

• URL: http://arxiv.org/abs/2511.19431v2
• Date: Tue, 25 Nov 2025 18:59:46 GMT
• Title: Cloud4D: Estimating Cloud Properties at a High Spatial and Temporal Resolution
• Abstract summary: We present Cloud4D, the first learning-based framework that reconstructs a physically consistent, four-dimensional cloud state.<n>Our system delivers an order-of-magnitude improvement in space-time resolution relative to state-of-the-art satellite measurements.
• Score: 8.745444691797067
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: There has been great progress in improving numerical weather prediction and climate models using machine learning. However, most global models act at a kilometer-scale, making it challenging to model individual clouds and factors such as extreme precipitation, wind gusts, turbulence, and surface irradiance. Therefore, there is a need to move towards higher-resolution models, which in turn require high-resolution real-world observations that current instruments struggle to obtain. We present Cloud4D, the first learning-based framework that reconstructs a physically consistent, four-dimensional cloud state using only synchronized ground-based cameras. Leveraging a homography-guided 2D-to-3D transformer, Cloud4D infers the full 3D distribution of liquid water content at 25 m spatial and 5 s temporal resolution. By tracking the 3D liquid water content retrievals over time, Cloud4D additionally estimates horizontal wind vectors. Across a two-month deployment comprising six skyward cameras, our system delivers an order-of-magnitude improvement in space-time resolution relative to state-of-the-art satellite measurements, while retaining single-digit relative error ($<10\%$) against collocated radar measurements. Code and data are available on our project page https://cloud4d.jacob-lin.com/.

Related papers

• SS4D: Native 4D Generative Model via Structured Spacetime Latents [50.29500511908054]
  We present SS4D, a native 4D generative model that synthesizes dynamic 3D objects directly from monocular video.<n>We train a generator directly on 4D data, achieving high fidelity, temporal coherence, and structural consistency.
  arXiv  Detail & Related papers  (2025-12-16T10:45:06Z)
• Global 3D Reconstruction of Clouds & Tropical Cyclones [0.5523946315455222]
  We introduce a new framework to translate 2D satellite imagery into 3D cloud maps of relevant cloud properties.<n>We show that we can - for the first time - create global instantaneous 3D cloud maps and accurately reconstruct the 3D structure of intense storms.
  arXiv  Detail & Related papers  (2025-11-06T19:47:03Z)
• 4D-ROLLS: 4D Radar Occupancy Learning via LiDAR Supervision [7.099012213719072]
  We propose 4D-ROLLS, the first weakly supervised occupancy estimation method for 4D radar using the LiDAR point cloud as the supervisory signal.<n>The model is seamlessly transferred to downstream tasks BEV segmentation and point cloud occupancy prediction.<n>The lightweight network enables 4D-ROLLS model to achieve fast inference speeds at about 30 Hz on a 4060 GPU.
  arXiv  Detail & Related papers  (2025-05-20T04:12:44Z)
• REHEARSE-3D: A Multi-modal Emulated Rain Dataset for 3D Point Cloud De-raining [0.5668912212306543]
  We release a new, large-scale, multi-modal emulated rain dataset, REHEARSE-3D, to promote research advancements in 3D point cloud de-raining.<n>First, it is the largest point-wise annotated dataset, and second, it is the only one with high-resolution LiDAR data enriched with 4D Radar point clouds.<n>We benchmark raindrop detection and removal in fused LiDAR and 4D Radar point clouds.
  arXiv  Detail & Related papers  (2025-04-30T14:43:38Z)
• Easi3R: Estimating Disentangled Motion from DUSt3R Without Training [69.51086319339662]
  We introduce Easi3R, a simple yet efficient training-free method for 4D reconstruction.<n>Our approach applies attention adaptation during inference, eliminating the need for from-scratch pre-training or network fine-tuning.<n>Our experiments on real-world dynamic videos demonstrate that our lightweight attention adaptation significantly outperforms previous state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-03-31T17:59:58Z)
• R2LDM: An Efficient 4D Radar Super-Resolution Framework Leveraging Diffusion Model [7.576799123793451]
  We introduce R2LDM, an innovative approach for generating dense and accurate 4D radar point clouds.<n>Instead of utilizing range images or bird's eye view (BEV) images, we represent both LiDAR and 4D radar point clouds using voxel features.<n>R2LDM effectively generates LiDAR-like point clouds from paired raw radar data.
  arXiv  Detail & Related papers  (2025-03-21T12:30:33Z)
• 3D Point Cloud Generation via Autoregressive Up-sampling [60.05226063558296]
  We introduce a pioneering autoregressive generative model for 3D point cloud generation.<n>Inspired by visual autoregressive modeling, we conceptualize point cloud generation as an autoregressive up-sampling process.<n>PointARU progressively refines 3D point clouds from coarse to fine scales.
  arXiv  Detail & Related papers  (2025-03-11T16:30:45Z)
• StarNet: Style-Aware 3D Point Cloud Generation [82.30389817015877]
  StarNet is able to reconstruct and generate high-fidelity and even 3D point clouds using a mapping network. Our framework achieves comparable state-of-the-art performance on various metrics in the point cloud reconstruction and generation tasks.
  arXiv  Detail & Related papers  (2023-03-28T08:21:44Z)
• A Conditional Point Diffusion-Refinement Paradigm for 3D Point Cloud Completion [69.32451612060214]
  Real-scanned 3D point clouds are often incomplete, and it is important to recover complete point clouds for downstream applications. Most existing point cloud completion methods use Chamfer Distance (CD) loss for training. We propose a novel Point Diffusion-Refinement (PDR) paradigm for point cloud completion.
  arXiv  Detail & Related papers  (2021-12-07T06:59:06Z)
• PC-DAN: Point Cloud based Deep Affinity Network for 3D Multi-Object Tracking (Accepted as an extended abstract in JRDB-ACT Workshop at CVPR21) [68.12101204123422]
  A point cloud is a dense compilation of spatial data in 3D coordinates. We propose a PointNet-based approach for 3D Multi-Object Tracking (MOT)
  arXiv  Detail & Related papers  (2021-06-03T05:36:39Z)
• Multi Projection Fusion for Real-time Semantic Segmentation of 3D LiDAR Point Clouds [2.924868086534434]
  This paper introduces a novel approach for 3D point cloud semantic segmentation that exploits multiple projections of the point cloud. Our Multi-Projection Fusion framework analyzes spherical and bird's-eye view projections using two separate highly-efficient 2D fully convolutional models.
  arXiv  Detail & Related papers  (2020-11-03T19:40:43Z)
• CloudCast: A Satellite-Based Dataset and Baseline for Forecasting Clouds [0.0]
  In this paper, we present a novel satellite-based dataset called CloudCast''. It consists of 70,080 images with 10 different cloud types for multiple layers of the atmosphere annotated on a pixel level. The spatial resolution of the dataset is 928 x 1530 pixels (3x3 km per pixel) with 15-min intervals between frames for the period 2017-01-01 to 2018-12-31.
  arXiv  Detail & Related papers  (2020-07-15T20:20:55Z)
• Pseudo-LiDAR Point Cloud Interpolation Based on 3D Motion Representation and Spatial Supervision [68.35777836993212]
  We propose a Pseudo-LiDAR point cloud network to generate temporally and spatially high-quality point cloud sequences. By exploiting the scene flow between point clouds, the proposed network is able to learn a more accurate representation of the 3D spatial motion relationship.
  arXiv  Detail & Related papers  (2020-06-20T03:11:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.