An iterative energy-based multimodal transformer for joint retrieval of wheat soil moisture, leaf area index, and plant height from Sentinel-1 and Sentinel-2 time series

• URL: http://arxiv.org/abs/2606.25174v1
• Date: Tue, 23 Jun 2026 21:05:53 GMT
• Title: An iterative energy-based multimodal transformer for joint retrieval of wheat soil moisture, leaf area index, and plant height from Sentinel-1 and Sentinel-2 time series
• Abstract summary: iEBT embeds multi-modal predictors within a shared sequence, produces an initial state estimate, and iteratively updates the target [SM, LAI, PH] vector.<n> Modality ablations confirmed that Sentinel-1 drives SM retrieval, while Sentinel-2 dominates LAI, whereas PH relies on combined structural-phenological signatures.
• Score: 1.86672057283442
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Field-scale retrieval of surface soil moisture (SM), leaf area index (LAI), and plant height (PH) is essential for precision agriculture, yet it remains an ill-posed inverse problem. Concurrent variations in soil moisture and canopy density generate substantial ambiguities in radar backscatter and spectral responses, which reduces the effectiveness of traditional feedforward regression models in heterogeneous smallholder cropping systems. This study presents the Iterative Energy-Based Transformer (iEBT) for the joint retrieval of coupled soil-canopy states from Sentinel-1 C-band SAR and Sentinel-2 multispectral time series. Instead of direct regression, iEBT embeds multi-modal predictors within a shared sequence, produces an initial state estimate, and iteratively updates the target [SM, LAI, PH] vector through normalized gradient descent to minimize a learned scalar compatibility energy function. Using 700 quality-controlled field measurements from Varanasi, India, iEBT achieved the highest learned-model performance on the random test split, with a four-seed mean R^2 of 0.854 \pm 0.012 (R_SM^2 = 0.841, R_LAI^2 = 0.905, R_PH^2 = 0.821). WCM and PROSAIL were retained as physically interpretable SAR and optical reference models for comparison. Modality ablations confirmed that Sentinel-1 drives SM retrieval, while Sentinel-2 dominates LAI, whereas PH relies on combined structural-phenological signatures. Crucially, the model's terminal energy functions as an uncalibrated post-retrieval quality diagnostic; screening the 10% highest-energy samples markedly reduced target level root-mean-square errors. While leave-one-campaign-out validation highlights persistent cross-season domain shift challenges due to localized management variations, compatibility-guided multimodal fusion offers a structured self-diagnostic path toward reliable biophysical parameter estimation

Related papers

• FLORO: A Multimodal Geospatial Foundation Model for Ecological Remote Sensing Across Sensors and Scales [65.4821703903285]
  FLORO is a multimodal foundation model designed to learn transferable representations from a small but highly diverse remote sensing corpus.<n> FLORO is pretrained using masked autoencoding on a heterogeneous combination of Sentinel-1, Sentinel-2, SkySAT imagery, elevation, and UAV-derived data.<n>We evaluated FLORO on the PANGAEA benchmark under a frozen-encoder protocol across scene classification, segmentation, and regression tasks.
  arXiv  Detail & Related papers  (2026-05-27T08:55:54Z)
• scHelix: Asymmetric Dual-Stream Integration via Explicit Gene-Level Disentanglement [84.88065404629079]
  scHelix is a dataset-adaptive framework that fundamentally changes how features are processed.<n> scHelix utilizes a dual-stream sparse diffusion encoder equipped with stop-gradient graph caching.<n> scHelix outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-05-18T15:55:13Z)
• Architecture-Agnostic Modality-Isolated Gated Fusion for Robust Multi-Modal Prostate MRI Segmentation [0.0]
  Multi-weighted prostate MRI combines T2-parametric (T2W), apparent diffusion coefficient (ADC), and high b-value diffusion-weighted (HBV) sequences.<n>In practice, the diffusion sequences are more frequently subject to acquisition variability, motion, and artifacts than T2W.<n>We propose Modality-Isolated Gated Fusion (MIGF) to maintain separate modality-specific encoding streams before a learned gating stage.
  arXiv  Detail & Related papers  (2026-04-12T15:54:21Z)
• SCALE:Scalable Conditional Atlas-Level Endpoint transport for virtual cell perturbation prediction [46.645872209565574]
  We present a specialized large-scale foundation model for virtual cell perturbation prediction.<n>First, we build a BioNeMo-based training and inference framework that substantially improves data throughput.<n>Second, we formulate perturbation prediction as conditional transport and implement it with a set-aware flow architecture.<n>Third, we evaluate the model on Tahoe-100M using a rigorous cell-level protocol centered on biologically meaningful metrics.
  arXiv  Detail & Related papers  (2026-03-18T05:51:08Z)
• Plug-and-Play Diffusion Meets ADMM: Dual-Variable Coupling for Robust Medical Image Reconstruction [45.25461515976432]
  Plug-and-Play diffusion prior (DP) frameworks have emerged as a powerful paradigm for imaging reconstruction.<n>We present a novel approach to resolving bias-hallucination trade-off, achieving state-of-the-art gradients with significantly accelerated convergence.
  arXiv  Detail & Related papers  (2026-02-26T16:58:43Z)
• Function-Space Decoupled Diffusion for Forward and Inverse Modeling in Carbon Capture and Storage [65.51149575007149]
  We present Fun-DDPS, a generative framework that combines function-space diffusion models with differentiable neural operator surrogates for both forward and inverse modeling.<n>Fun-DDPS produces physically consistent realizations free from the high-frequency artifacts observed in joint-state baselines.
  arXiv  Detail & Related papers  (2026-02-12T18:58:12Z)
• Stage-Specific Benchmarking of Deep Learning Models for Glioblastoma Follow-Up MRI [1.1458853556386799]
  We present the first stage-specific, cross-sectional benchmarking of deep learning models for follow-up MRI.<n>We analyze different post-RT scans independently to test whether architecture performance depends on time-point.<n>These results establish a stage-aware benchmark and motivate future work incorporating longitudinal modeling, multi-sequence MRI, and larger multi-center cohorts.
  arXiv  Detail & Related papers  (2025-11-23T19:38:03Z)
• Transformers vs. Recurrent Models for Estimating Forest Gross Primary Production [32.81344785160551]
  Remote sensing offers an alternative to single-sensor spectral indices and statistical models.<n>Recent advances in deep learning (DL) and data fusion offer new opportunities to better represent the temporal dynamics of vegetation processes.<n>Here, we explore the performance of two representative models for predicting Gross Primary Production.
  arXiv  Detail & Related papers  (2025-11-14T21:18:01Z)
• Causal Time Series Modeling of Supraglacial Lake Evolution in Greenland under Distribution Shift [2.5551933647600693]
  Causal modeling offers a principled foundation for uncovering stable, invariant relationships in time-series data.<n>We propose RIC-TSC, a regionally-informed causal time-series classification framework that embeds lag-aware causal discovery directly into sequence modeling.
  arXiv  Detail & Related papers  (2025-10-17T03:06:08Z)
• Field-scale soil moisture estimated from Sentinel-1 SAR data using a knowledge-guided deep learning approach [6.595840767689357]
  This research presents preliminary efforts for developing a knowledge-guided deep learning approach.<n>It integrates the water cloud model (WCM) principles into a long short-term memory (LSTM) model.<n>Our proposed approach builds upon the soil backscatter coefficients isolated from the total backscatter, together with Landsat-resolution vegetation information and surface characteristics.<n>Results showed the proposed approach reduced SM retrieval by 0.02 m$3$/m$3$ and achieved correlation coefficients (R) of up to 0.64 in areas with varying vegetation cover and surface conditions.
  arXiv  Detail & Related papers  (2025-05-01T03:12:25Z)
• Multi-Source and Test-Time Domain Adaptation on Multivariate Signals using Spatio-Temporal Monge Alignment [59.75420353684495]
  Machine learning applications on signals such as computer vision or biomedical data often face challenges due to the variability that exists across hardware devices or session recordings. In this work, we propose Spatio-Temporal Monge Alignment (STMA) to mitigate these variabilities. We show that STMA leads to significant and consistent performance gains between datasets acquired with very different settings.
  arXiv  Detail & Related papers  (2024-07-19T13:33:38Z)
• Evaluation of the potential of Near Infrared Hyperspectral Imaging for monitoring the invasive brown marmorated stink bug [53.682955739083056]
  The brown marmorated stink bug (BMSB), Halyomorpha halys, is an invasive insect pest of global importance that damages several crops. The present study consists in a preliminary evaluation at the laboratory level of Near Infrared Hyperspectral Imaging (NIR-HSI) as a possible technology to detect BMSB specimens.
  arXiv  Detail & Related papers  (2023-01-19T11:37:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.