DeepShade: Enable Shade Simulation by Text-conditioned Image Generation

• URL: http://arxiv.org/abs/2507.12103v4
• Date: Fri, 05 Sep 2025 22:26:45 GMT
• Title: DeepShade: Enable Shade Simulation by Text-conditioned Image Generation
• Abstract summary: Current routing systems fail to incorporate shade information due to the difficulty of estimating shades directly from satellite imagery.<n>We build an extensive dataset covering diverse longitude-latitude regions, varying levels of building density, and different urban layouts.<n>We propose the DeepShade, a diffusion-based model designed to learn and synthesize shade variations over time.
• Score: 30.44820766374578
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Heatwaves pose a significant threat to public health, especially as global warming intensifies. However, current routing systems (e.g., online maps) fail to incorporate shade information due to the difficulty of estimating shades directly from noisy satellite imagery and the limited availability of training data for generative models. In this paper, we address these challenges through two main contributions. First, we build an extensive dataset covering diverse longitude-latitude regions, varying levels of building density, and different urban layouts. Leveraging Blender-based 3D simulations alongside building outlines, we capture building shadows under various solar zenith angles throughout the year and at different times of day. These simulated shadows are aligned with satellite images, providing a rich resource for learning shade patterns. Second, we propose the DeepShade, a diffusion-based model designed to learn and synthesize shade variations over time. It emphasizes the nuance of edge features by jointly considering RGB with the Canny edge layer, and incorporates contrastive learning to capture the temporal change rules of shade. Then, by conditioning on textual descriptions of known conditions (e.g., time of day, solar angles), our framework provides improved performance in generating shade images. We demonstrate the utility of our approach by using our shade predictions to calculate shade ratios for real-world route planning in Tempe, Arizona. We believe this work will benefit society by providing a reference for urban planning in extreme heat weather and its potential practical applications in the environment.

Related papers

• GridFlow: Structured Latent Flow for Seamless City-Scale 3D Point Cloud Generation [33.289877090164275]
  We present ours, a framework that generates colored point clouds at city scale conditioned on satellite imagery, semantic segmentation maps, and digital surface models.<n> Experiments show that ours outperforms adapted point cloud generation baselines across all geometry metrics.
  arXiv  Detail & Related papers  (2026-08-30T13:39:36Z)
• ShadeBench: A Benchmark Dataset for Building Shade Simulation in Sustainable Society [30.44820766374578]
  ShadeBench is a comprehensive dataset and benchmark for urban shade understanding.<n>Built upon this multimodal dataset, ShadeBench supports a range of downstream tasks, including shade generation, shade segmentation, and 3D building reconstruction.
  arXiv  Detail & Related papers  (2026-05-19T21:28:02Z)
• Joint Shadow Generation and Relighting via Light-Geometry Interaction Maps [51.82696819319878]
  We propose Light-Geometry Interaction maps, a novel representation that encodes light-aware occlusion from monocular depth.<n>LGI captures essential light-shadow interactions reliably and accurately, computed from off-the-shelf 2.5D depth map predictions.<n>By embedding LGI into a bridge-matching generative backbone, we reduce ambiguity and enforce physically consistent light-shadow reasoning.
  arXiv  Detail & Related papers  (2026-02-25T11:47:26Z)
• WeatherDiffusion: Weather-Guided Diffusion Model for Forward and Inverse Rendering [40.94600501568197]
  WeatherDiffusion is a diffusion-based framework for forward and inverse rendering on autonomous driving scenes.<n>Our method enables authentic estimation of material properties, scene geometry, and lighting, and further supports controllable weather and illumination editing.
  arXiv  Detail & Related papers  (2025-08-09T13:29:39Z)
• The Devil is in the Darkness: Diffusion-Based Nighttime Dehazing Anchored in Brightness Perception [58.895000127068194]
  We introduce the Diffusion-Based Nighttime Dehazing framework, which excels in both data synthesis and lighting reconstruction.<n>We propose a restoration model that integrates a pre-trained diffusion model guided by a brightness perception network.<n>Experiments validate our dataset's utility and the model's superior performance in joint haze removal and brightness mapping.
  arXiv  Detail & Related papers  (2025-06-03T03:21:13Z)
• Shaded Route Planning Using Active Segmentation and Identification of Satellite Images [2.2162879952427343]
  Vulnerable groups, especially pedestrians and cyclists on sun-exposed sidewalks, motivate the development of a route planning method. This paper is the first to introduce a pipeline that utilizes segmentation foundation models to extract shaded areas from satellite images. These areas are then integrated into a multi-layered road map, enabling users to customize routes based on a balance between distance and shade exposure. This system is already implemented online, with a video demonstration, and will be specifically adapted to assist travelers during the 2024 Olympic Games in Paris.
  arXiv  Detail & Related papers  (2024-07-18T16:57:11Z)
• LidaRF: Delving into Lidar for Neural Radiance Field on Street Scenes [73.65115834242866]
  Photorealistic simulation plays a crucial role in applications such as autonomous driving. However, reconstruction quality suffers on street scenes due to collinear camera motions and sparser samplings at higher speeds. We propose several insights that allow a better utilization of Lidar data to improve NeRF quality on street scenes.
  arXiv  Detail & Related papers  (2024-05-01T23:07:12Z)
• Deep Umbra: A Generative Approach for Sunlight Access Computation in Urban Spaces [4.680816326435908]
  We propose a novel framework that enables the quantifying of sunlight access and shadows at a global scale. Our framework is based on a conditional generative adversarial network that considers the physical form of cities. We use data from seven different cities to train our model, and show, through an extensive set of experiments, its low overall RMSE. We contribute a set of case studies and a comprehensive dataset with sunlight access information for more than 100 cities across six continents.
  arXiv  Detail & Related papers  (2024-02-27T03:05:05Z)
• Sat2Scene: 3D Urban Scene Generation from Satellite Images with Diffusion [77.34078223594686]
  We propose a novel architecture for direct 3D scene generation by introducing diffusion models into 3D sparse representations and combining them with neural rendering techniques. Specifically, our approach generates texture colors at the point level for a given geometry using a 3D diffusion model first, which is then transformed into a scene representation in a feed-forward manner. Experiments in two city-scale datasets show that our model demonstrates proficiency in generating photo-realistic street-view image sequences and cross-view urban scenes from satellite imagery.
  arXiv  Detail & Related papers  (2024-01-19T16:15:37Z)
• Incorporating Season and Solar Specificity into Renderings made by a NeRF Architecture using Satellite Images [0.0]
  We show how one can make the renderings season-specific using a NeRF-based framework. NeRF can render seasonal features independently of viewing angle and solar angle while still being able to render shadows. We show the performance of our network on eight Areas of Interest containing images captured by the Maxar WorldView-3 satellite.
  arXiv  Detail & Related papers  (2023-08-02T16:30:18Z)
• UrbanIR: Large-Scale Urban Scene Inverse Rendering from a Single Video [22.613946218766802]
  We present UrbanIR, a new inverse graphics model that enables realistic, free-viewpoint renderings of scenes under various lighting conditions with a single video.<n>It accurately infers shape, albedo, visibility, and sun and sky illumination from wide-baseline videos, such as those from car-mounted cameras.<n>UrbanIR addresses these issues with novel losses that reduce errors in inverse graphics inference and rendering artifacts.
  arXiv  Detail & Related papers  (2023-06-15T17:59:59Z)
• Exploring the Application of Large-scale Pre-trained Models on Adverse Weather Removal [97.53040662243768]
  We propose a CLIP embedding module to make the network handle different weather conditions adaptively. This module integrates the sample specific weather prior extracted by CLIP image encoder together with the distribution specific information learned by a set of parameters.
  arXiv  Detail & Related papers  (2023-06-15T10:06:13Z)
• GECCO: Geometrically-Conditioned Point Diffusion Models [60.28388617034254]
  Diffusion models generating images conditionally on text have recently made a splash far beyond the computer vision community. Here, we tackle the related problem of generating point clouds, both unconditionally, and conditionally with images. For the latter, we introduce a novel geometrically-motivated conditioning scheme based on projecting sparse image features into the point cloud.
  arXiv  Detail & Related papers  (2023-03-10T13:45:44Z)
• Geometry-aware Single-image Full-body Human Relighting [37.381122678376805]
  Single-image human relighting aims to relight a target human under new lighting conditions by decomposing the input image into albedo, shape and lighting. Previous methods suffer from both the entanglement between albedo and lighting and the lack of hard shadows. Our framework is able to generate photo-realistic high-frequency shadows such as cast shadows under challenging lighting conditions.
  arXiv  Detail & Related papers  (2022-07-11T10:21:02Z)
• A Shading-Guided Generative Implicit Model for Shape-Accurate 3D-Aware Image Synthesis [163.96778522283967]
  We propose a shading-guided generative implicit model that is able to learn a starkly improved shape representation. An accurate 3D shape should also yield a realistic rendering under different lighting conditions. Our experiments on multiple datasets show that the proposed approach achieves photorealistic 3D-aware image synthesis.
  arXiv  Detail & Related papers  (2021-10-29T10:53:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.