Physics-Guided VLM Priors for All-Cloud Removal

• URL: http://arxiv.org/abs/2603.07074v1
• Date: Sat, 07 Mar 2026 07:07:46 GMT
• Title: Physics-Guided VLM Priors for All-Cloud Removal
• Abstract summary: Cloud removal is a fundamental challenge in optical remote sensing due to the heterogeneous degradation.<n>Existing pipelines separate thin-cloud correction from thick-cloud reconstruction, requiring explicit cloud-type decisions.<n>We propose Physical-VLM All-Cloud Removal (PhyVLM-CR) that integrates the semantic capability of Vision-Language Model (VLM) into a physical restoration model.
• Score: 5.824884352725767
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Cloud removal is a fundamental challenge in optical remote sensing due to the heterogeneous degradation. Thin clouds distort radiometry via partial transmission, while thick clouds occlude the surface. Existing pipelines separate thin-cloud correction from thick-cloud reconstruction, requiring explicit cloud-type decisions and often leading to error accumulation and discontinuities in mixed-cloud scenes. Therefore, a novel approach named Physical-VLM All-Cloud Removal (PhyVLM-CR) that integrates the semantic capability of Vision-Language Model (VLM) into a physical restoration model, achieving high-fidelity unified cloud removal. Specifically, the cognitive prior from a VLM (e.g., Qwen) is transformed into physical scattering parameters and a hallucination confidence map. Leveraging this confidence map as a continuous soft gate, our method achieves a unified restoration via adaptive weighting: it prioritizes physical inversion in high-transmission regions to preserve radiometric fidelity, while seamlessly transitioning to temporal reference reconstruction in low-confidence occluded areas. This mechanism eliminates the need for explicit boundary delineation, ensuring a coherent removal across heterogeneous cloud covers. Experiments on real-world Sentinel-2 surface reflectance imagery confirm that our approach achieves a remarkable balance between cloud removal and content preservation, delivering hallucination-free results with substantially improved quantitative accuracy compared to existing methods.

Related papers

• EraseSAE: Surgical Concept Erasure in Text-to-Video Diffusion Models via Sparse Autoencoders [103.46709005379591]
  Concept erasure offers a principled remedy by removing unwanted semantics from pretrained models.<n>Existing approaches typically operate at a coarse quality, misaligned with the finegrained, distributed nature of concept representations.<n>We propose EraseSAE, a novel framework that leverages sparse autoencoders to achieve surgical concept erasure.
  arXiv  Detail & Related papers  (2026-09-03T10:23:37Z)
• SADER: Structure-Aware Diffusion Framework with DEterministic Resampling for Multi-Temporal Remote Sensing Cloud Removal [24.84158171817714]
  Diffusion-based models have emerged as a dominant paradigm for remote sensing cloud removal.<n>We propose SADER, a structure-aware diffusion framework for multi-temporal remote sensing cloud removal.<n>Experiments on multiple multi-temporal datasets demonstrate that SADER consistently outperforms state-of-the-art cloud removal methods.
  arXiv  Detail & Related papers  (2026-01-31T06:06:18Z)
• LAFR: Efficient Diffusion-based Blind Face Restoration via Latent Codebook Alignment Adapter [52.93785843453579]
  Blind face restoration from low-quality (LQ) images is a challenging task that requires high-fidelity image reconstruction and the preservation of facial identity.<n>We propose LAFR, a novel codebook-based latent space adapter that aligns the latent distribution of LQ images with that of HQ counterparts.<n>We show that lightweight finetuning of diffusion prior on just 0.9% of FFHQ dataset is sufficient to achieve results comparable to state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-05-29T14:11:16Z)
• When Cloud Removal Meets Diffusion Model in Remote Sensing [4.824120664293887]
  We propose DC4CR (Diffusion Control for Cloud Removal), a novel framework for cloud removal in remote sensing imagery.<n>Our method introduces prompt-driven control, allowing selective removal of thin and thick clouds without relying on pre-generated cloud masks.<n>Experiments on the RICE and CUHK-CR datasets demonstrate state-of-the-art performance, achieving superior cloud removal across diverse conditions.
  arXiv  Detail & Related papers  (2025-04-21T00:56:57Z)
• Patch-GAN Transfer Learning with Reconstructive Models for Cloud Removal [17.690698736544626]
  Cloud removal plays a crucial role in enhancing remote sensing image analysis, yet accurately reconstructing cloud-obscured regions remains a significant challenge.<n>Recent advancements in generative models have made the generation of realistic images increasingly accessible.<n>We propose a deep transfer learning approach built on a generative adversarial network (GAN) framework to explore the potential of the novel masked autoencoder (MAE) image reconstruction model in cloud removal.
  arXiv  Detail & Related papers  (2025-01-09T14:19:46Z)
• Point Cloud Resampling with Learnable Heat Diffusion [58.050130177241186]
  We propose a learnable heat diffusion framework for point cloud resampling. Unlike previous diffusion models with a fixed prior, the adaptive conditional prior selectively preserves geometric features of the point cloud.
  arXiv  Detail & Related papers  (2024-11-21T13:44:18Z)
• Rendering-Oriented 3D Point Cloud Attribute Compression using Sparse Tensor-based Transformer [52.40992954884257]
  3D visualization techniques have fundamentally transformed how we interact with digital content.<n>Massive data size of point clouds presents significant challenges in data compression.<n>We propose an end-to-end deep learning framework that seamlessly integrates PCAC with differentiable rendering.
  arXiv  Detail & Related papers  (2024-11-12T16:12:51Z)
• IDF-CR: Iterative Diffusion Process for Divide-and-Conquer Cloud Removal in Remote-sensing Images [55.40601468843028]
  We present an iterative diffusion process for cloud removal (IDF-CR) IDF-CR is divided into two-stage models that address pixel space and latent space. In the latent space stage, the diffusion model transforms low-quality cloud removal into high-quality clean output.
  arXiv  Detail & Related papers  (2024-03-18T15:23:48Z)
• UnCRtainTS: Uncertainty Quantification for Cloud Removal in Optical Satellite Time Series [19.32220113046804]
  We introduce UnCRtainTS, a method for multi-temporal cloud removal combining a novel attention-based architecture. We show how the well-calibrated predicted uncertainties enable a precise control of the reconstruction quality.
  arXiv  Detail & Related papers  (2023-04-11T19:27:18Z)
• Pseudo-LiDAR Point Cloud Interpolation Based on 3D Motion Representation and Spatial Supervision [68.35777836993212]
  We propose a Pseudo-LiDAR point cloud network to generate temporally and spatially high-quality point cloud sequences. By exploiting the scene flow between point clouds, the proposed network is able to learn a more accurate representation of the 3D spatial motion relationship.
  arXiv  Detail & Related papers  (2020-06-20T03:11:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.