GeoSET: Generalist Foundation Model for SAR-to-EO Image Translation
Abstract Overview
GeoSET is presented as a generalist foundation model for SAR-to-EO image translation under a pretrain-once, adapt-many framework. The paper builds a heterogeneous pretraining corpus of 3,204,744 filtered SAR-EO pairs curated from more than 10 million SAR observations, spanning multiple sensors, resolutions, and regions. Methodologically, it combines a speckle-robust SAR encoder with conditional flow-based pretraining of a repurposed pretrained image generator, then adapts the parent model to downstream datasets using either full fine-tuning or LoRA. The study evaluates the approach on six downstream benchmarks and emphasizes distributional realism and perceptual fidelity through metrics such as FID and DISTS.
Novelty
The paper introduces the first generalist SAR-to-EO translation model trained once on a large heterogeneous paired corpus and then adapted across downstream datasets under a common protocol. Its key technical contributions include a curated multi-source SAR-EO pretraining corpus, a speckle-robust SAR encoder compatible with a pretrained latent space, and the repurposing of a pretrained flow generator for spatial SAR conditioning.
Results
Across six downstream benchmarks, GeoSET achieves state-of-the-art FID scores and top DISTS performance on most datasets across full fine-tuning and LoRA modes. Downstream LoRA adaptation updates only approximately 0.60% of generator parameters in about one hour per dataset while performing competitively with full fine-tuning. Ablations confirm cumulative improvements from large-scale pretraining, SAR-specific encoding, and domain-aware speckle augmentation.
Key Points
- GeoSET establishes a pretrain-once, adapt-many framework for SAR-to-EO translation using a single generalist parent model rather than isolated dataset-specific models.
- The framework incorporates a multi-source pretraining corpus of over 3.2 million filtered pairs alongside a speckle-robust SAR encoder aligned with a pretrained image autoencoder.
- Empirical benchmarks across six datasets demonstrate superior distributional realism and perceptual metrics, with LoRA providing efficient downstream adaptation with minimal trainable parameters and storage.