CarbonCLIP: Enhance Carbon Prediction from Satellite Imagery via Integrated Street-View Semantics and Temporal Context Training

• URL: http://arxiv.org/abs/2607.07292v1
• Date: Wed, 08 Jul 2026 11:33:35 GMT
• Title: CarbonCLIP: Enhance Carbon Prediction from Satellite Imagery via Integrated Street-View Semantics and Temporal Context Training
• Abstract summary: CarbonCLIP is a task-oriented multimodal distillation framework that improves satellite-based carbon emission prediction.<n>Unlike conventional methods that rely on static visual features, CarbonCLIP explicitly bridges the gap between top-down satellite views and ground-level human activities.<n>CarbonCLIP requires multimodal data only during the pretraining phase; during inference, it relies solely on satellite imagery.
• Score: 41.99844472131922
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Accurately estimating urban carbon emissions is critical for sustainable urban planning, yet many existing approaches remain difficult to apply consistently across cities due to data-source heterogeneity and the lack of fine-grained semantic-temporal context in remote sensing data. We propose CarbonCLIP, a task-oriented multimodal distillation framework that improves satellite-based carbon emission prediction by transferring contextual knowledge into a unified satellite representation through dual-branch contrastive learning. Unlike conventional methods that rely on static visual features, CarbonCLIP explicitly bridges the gap between top-down satellite views and ground-level human activities. Specifically, the spatial branch uses fine-grained textual descriptions automatically generated from street-view images by Large Multimodal Models (LMMs) to provide semantic priors reflecting building functions, infrastructure, and urban activities, while the temporal branch employs a month encoder to encode temporal priors associated with monthly emission variation. CarbonCLIP requires multimodal data only during the pretraining phase; during inference, it relies solely on satellite imagery, thereby supporting scalable deployment when ground-level data are unavailable at inference. Experiments on Beijing and Singapore demonstrate that CarbonCLIP outperforms baselines in both study cities. The results validate that our method effectively transfers multimodal knowledge into satellite representations, offering a robust solution for satellite-based urban carbon modeling.

Related papers

• CarbonBench: A Global Benchmark for Upscaling of Carbon Fluxes Using Zero-Shot Learning [9.490900991393636]
  CarbonBench is a benchmark for zero-shot spatial transfer in carbon flux upscaling.<n>It comprises over 1.3 million daily observations from 567 flux tower sites globally.<n>By bridging machine learning methodologies and Earth system science, CarbonBench aims to enable comparison of transfer learning methods.
  arXiv  Detail & Related papers  (2026-03-10T16:33:28Z)
• WalkCLIP: Multimodal Learning for Urban Walkability Prediction [4.403122905236942]
  Urban walkability is a cornerstone of public health, sustainability, and quality of life.<n>Recent studies have used satellite imagery, street view imagery, or population indicators to estimate walkability.<n>We introduce WalkCLIP, a framework that integrates these complementary viewpoints to predict urban walkability.
  arXiv  Detail & Related papers  (2025-11-26T22:15:33Z)
• Scaling Up Occupancy-centric Driving Scene Generation: Dataset and Method [54.461213497603154]
  Occupancy-centric methods have recently achieved state-of-the-art results by offering consistent conditioning across frames and modalities.<n>Nuplan-Occ is the largest occupancy dataset to date, constructed from the widely used Nuplan benchmark.<n>We develop a unified framework that jointly synthesizes high-quality occupancy, multi-view videos, and LiDAR point clouds.
  arXiv  Detail & Related papers  (2025-10-27T03:52:45Z)
• Multimodal Contrastive Learning of Urban Space Representations from POI Data [2.695321027513952]
  CaLLiPer (Contrastive Language-Location Pre-training) is a representation learning model that embeds continuous urban spaces into vector representations. We validate CaLLiPer's effectiveness by applying it to learning urban space representations in London, UK.
  arXiv  Detail & Related papers  (2024-11-09T16:24:07Z)
• UB-FineNet: Urban Building Fine-grained Classification Network for Open-access Satellite Images [7.435848987082052]
  We propose a deep network approach to fine-grained classification of urban buildings using open-access satellite images. A new fine-grained classification network with Category Information Balancing Module (CIBM) and Contrastive Supervision (CS) technique is proposed to mitigate the problem of class imbalance. Experiments on Hong Kong data set with 11 fine building types revealed promising classification results with a mean Top-1 accuracy of 60.45%.
  arXiv  Detail & Related papers  (2024-03-04T15:40:31Z)
• Exploring the Application of Large-scale Pre-trained Models on Adverse Weather Removal [97.53040662243768]
  We propose a CLIP embedding module to make the network handle different weather conditions adaptively. This module integrates the sample specific weather prior extracted by CLIP image encoder together with the distribution specific information learned by a set of parameters.
  arXiv  Detail & Related papers  (2023-06-15T10:06:13Z)
• SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagery [74.82821342249039]
  We present SatMAE, a pre-training framework for temporal or multi-spectral satellite imagery based on Masked Autoencoder (MAE) To leverage temporal information, we include a temporal embedding along with independently masking image patches across time.
  arXiv  Detail & Related papers  (2022-07-17T01:35:29Z)
• Measuring the Carbon Intensity of AI in Cloud Instances [91.28501520271972]
  We provide a framework for measuring software carbon intensity, and propose to measure operational carbon emissions. We evaluate a suite of approaches for reducing emissions on the Microsoft Azure cloud compute platform.
  arXiv  Detail & Related papers  (2022-06-10T17:04:04Z)
• Spatio-Temporal Graph Few-Shot Learning with Cross-City Knowledge Transfer [58.6106391721944]
  Cross-city knowledge has shown its promise, where the model learned from data-sufficient cities is leveraged to benefit the learning process of data-scarce cities. We propose a model-agnostic few-shot learning framework for S-temporal graph called ST-GFSL. We conduct comprehensive experiments on four traffic speed prediction benchmarks and the results demonstrate the effectiveness of ST-GFSL compared with state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-05-27T12:46:52Z)
• A Semantic Segmentation Network for Urban-Scale Building Footprint Extraction Using RGB Satellite Imagery [1.9400948599830012]
  Urban areas consume over two-thirds of the world's energy and account for more than 70 percent of global CO2 emissions. We propose a modified DeeplabV3+ module with a Dilated ResNet backbone to generate masks of building footprints from only three-channel RGB satellite imagery. We achieve state-of-the-art performance across three standard benchmarks and demonstrate that our method is agnostic to the scale, resolution, and urban density of satellite imagery.
  arXiv  Detail & Related papers  (2021-04-02T22:32:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.