Physics informed Transformer-VAE for biophysical parameter estimation: PROSAIL model inversion in Sentinel-2 imagery

• URL: http://arxiv.org/abs/2511.10387v3
• Date: Mon, 17 Nov 2025 17:20:09 GMT
• Title: Physics informed Transformer-VAE for biophysical parameter estimation: PROSAIL model inversion in Sentinel-2 imagery
• Authors: Prince Mensah, Pelumi Victor Aderinto, Ibrahim Salihu Yusuf, Arnu Pretorius,
• Abstract summary: We propose a physics-informed Transformer-VAE architecture to invert the PROSAIL radiative transfer model for simultaneous estimation of key canopy parameters from Sentinel-2 data.<n>Our model is trained exclusively on simulated data, yet achieves performance on par with state-of-the-art methods that utilize real imagery.
• Score: 4.19792535128126
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Accurate retrieval of vegetation biophysical variables from satellite imagery is crucial for ecosystem monitoring and agricultural management. In this work, we propose a physics-informed Transformer-VAE architecture to invert the PROSAIL radiative transfer model for simultaneous estimation of key canopy parameters from Sentinel-2 data. Unlike previous hybrid approaches that require real satellite images for self-supevised training. Our model is trained exclusively on simulated data, yet achieves performance on par with state-of-the-art methods that utilize real imagery. The Transformer-VAE incorporates the PROSAIL model as a differentiable physical decoder, ensuring that inferred latent variables correspond to physically plausible leaf and canopy properties. We demonstrate retrieval of leaf area index (LAI) and canopy chlorophyll content (CCC) on real-world field datasets (FRM4Veg and BelSAR) with accuracy comparable to models trained with real Sentinel-2 data. Our method requires no in-situ labels or calibration on real images, offering a cost-effective and self-supervised solution for global vegetation monitoring. The proposed approach illustrates how integrating physical models with advanced deep networks can improve the inversion of RTMs, opening new prospects for large-scale, physically-constrained remote sensing of vegetation traits.

Related papers

• Traj-Transformer: Diffusion Models with Transformer for GPS Trajectory Generation [15.689474391811734]
  We propose Trajectory Transformer, a novel model that employs a transformer backbone for both conditional information embedding and noise prediction.<n>Experiments on two real-world datasets demonstrate that Tray Transformer significantly enhances generation quality and effectively alleviates the issues observed in prior approaches.
  arXiv  Detail & Related papers  (2025-10-07T05:41:09Z)
• ASTRA: A Scene-aware TRAnsformer-based model for trajectory prediction [15.624698974735654]
  ASTRA (A Scene-aware TRAnsformer-based model for trajectory prediction) is a light-weight pedestrian trajectory forecasting model.<n>We utilise a U-Net-based feature extractor, via its latent vector representation, to capture scene representations and a graph-aware transformer encoder for capturing social interactions.
  arXiv  Detail & Related papers  (2025-01-16T23:28:30Z)
• HorGait: A Hybrid Model for Accurate Gait Recognition in LiDAR Point Cloud Planar Projections [8.56443762544299]
  HorGait is a hybrid model with a Transformer architecture for gait recognition on the planar projection of 3D point clouds from LiDAR. It achieves state-of-the-art performance among Transformer architecture methods on the SUSTech1K dataset.
  arXiv  Detail & Related papers  (2024-10-11T02:12:41Z)
• S^2Former-OR: Single-Stage Bi-Modal Transformer for Scene Graph Generation in OR [50.435592120607815]
  Scene graph generation (SGG) of surgical procedures is crucial in enhancing holistically cognitive intelligence in the operating room (OR) Previous works have primarily relied on multi-stage learning, where the generated semantic scene graphs depend on intermediate processes with pose estimation and object detection. In this study, we introduce a novel single-stage bi-modal transformer framework for SGG in the OR, termed S2Former-OR.
  arXiv  Detail & Related papers  (2024-02-22T11:40:49Z)
• Physics-Driven Turbulence Image Restoration with Stochastic Refinement [80.79900297089176]
  Image distortion by atmospheric turbulence is a critical problem in long-range optical imaging systems. Fast and physics-grounded simulation tools have been introduced to help the deep-learning models adapt to real-world turbulence conditions. This paper proposes the Physics-integrated Restoration Network (PiRN) to help the network to disentangle theity from the degradation and the underlying image.
  arXiv  Detail & Related papers  (2023-07-20T05:49:21Z)
• Inference over radiative transfer models using variational and expectation maximization methods [9.73020420215473]
  We introduce two computational techniques to infer not only point estimates of biophysical parameters but also their joint distribution. One of them is based on a variational autoencoder approach and the second one is based on a Monte Carlo Expectation Maximization scheme. We analyze the performance of the two approaches for modeling and inferring the distribution of three key biophysical parameters for quantifying the terrestrial biosphere.
  arXiv  Detail & Related papers  (2022-04-07T10:33:51Z)
• ViTAEv2: Vision Transformer Advanced by Exploring Inductive Bias for Image Recognition and Beyond [76.35955924137986]
  We propose a Vision Transformer Advanced by Exploring intrinsic IB from convolutions, i.e., ViTAE. ViTAE has several spatial pyramid reduction modules to downsample and embed the input image into tokens with rich multi-scale context. We obtain the state-of-the-art classification performance, i.e., 88.5% Top-1 classification accuracy on ImageNet validation set and the best 91.2% Top-1 accuracy on ImageNet real validation set.
  arXiv  Detail & Related papers  (2022-02-21T10:40:05Z)
• CSformer: Bridging Convolution and Transformer for Compressive Sensing [65.22377493627687]
  This paper proposes a hybrid framework that integrates the advantages of leveraging detailed spatial information from CNN and the global context provided by transformer for enhanced representation learning. The proposed approach is an end-to-end compressive image sensing method, composed of adaptive sampling and recovery. The experimental results demonstrate the effectiveness of the dedicated transformer-based architecture for compressive sensing.
  arXiv  Detail & Related papers  (2021-12-31T04:37:11Z)
• Learning Generative Vision Transformer with Energy-Based Latent Space for Saliency Prediction [51.80191416661064]
  We propose a novel vision transformer with latent variables following an informative energy-based prior for salient object detection. Both the vision transformer network and the energy-based prior model are jointly trained via Markov chain Monte Carlo-based maximum likelihood estimation. With the generative vision transformer, we can easily obtain a pixel-wise uncertainty map from an image, which indicates the model confidence in predicting saliency from the image.
  arXiv  Detail & Related papers  (2021-12-27T06:04:33Z)
• FoveaTer: Foveated Transformer for Image Classification [8.207403859762044]
  We propose foveated Transformer (FoveaTer) model, which uses pooling regions and saccadic movements to perform object classification tasks. We construct an ensemble model using our proposed model and unfoveated model, achieving an accuracy 1.36% below the unfoveated model with 22% computational savings.
  arXiv  Detail & Related papers  (2021-05-29T01:54:33Z)
• Visual Saliency Transformer [127.33678448761599]
  We develop a novel unified model based on a pure transformer, Visual Saliency Transformer (VST), for both RGB and RGB-D salient object detection (SOD) It takes image patches as inputs and leverages the transformer to propagate global contexts among image patches. Experimental results show that our model outperforms existing state-of-the-art results on both RGB and RGB-D SOD benchmark datasets.
  arXiv  Detail & Related papers  (2021-04-25T08:24:06Z)
• Estimating Crop Primary Productivity with Sentinel-2 and Landsat 8 using Machine Learning Methods Trained with Radiative Transfer Simulations [58.17039841385472]
  We take advantage of all parallel developments in mechanistic modeling and satellite data availability for advanced monitoring of crop productivity. Our model successfully estimates gross primary productivity across a variety of C3 crop types and environmental conditions even though it does not use any local information from the corresponding sites. This highlights its potential to map crop productivity from new satellite sensors at a global scale with the help of current Earth observation cloud computing platforms.
  arXiv  Detail & Related papers  (2020-12-07T16:23:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.