Spectrally Distilled Representations Aligned with Instruction-Augmented LLMs for Satellite Imagery

• URL: http://arxiv.org/abs/2602.22613v1
• Date: Thu, 26 Feb 2026 04:34:47 GMT
• Title: Spectrally Distilled Representations Aligned with Instruction-Augmented LLMs for Satellite Imagery
• Abstract summary: Vision-language foundation models (VLFMs) promise zero-shot and retrieval understanding for Earth observation.<n>We present SATtxt, a spectrum-aware VLFM that operates with RGB inputs only at inference.<n>Across EuroSAT, BigEarthNet, and ForestNet, SATtxt improves zero-shot classification on average by 4.2%, retrieval by 5.9%, and linear probing by 2.7% over baselines.
• Score: 37.61154732856327
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision-language foundation models (VLFMs) promise zero-shot and retrieval understanding for Earth observation. While operational satellite systems often lack full multi-spectral coverage, making RGB-only inference highly desirable for scalable deployment, the adoption of VLFMs for satellite imagery remains hindered by two factors: (1) multi-spectral inputs are informative but difficult to exploit consistently due to band redundancy and misalignment; and (2) CLIP-style text encoders limit semantic expressiveness and weaken fine-grained alignment. We present SATtxt, a spectrum-aware VLFM that operates with RGB inputs only at inference while retaining spectral cues learned during training. Our framework comprises two stages. First, Spectral Representation Distillation transfers spectral priors from a frozen multi-spectral teacher to an RGB student via a lightweight projector. Second, Spectrally Grounded Alignment with Instruction-Augmented LLMs bridges the distilled visual space and an expressive LLM embedding space. Across EuroSAT, BigEarthNet, and ForestNet, SATtxt improves zero-shot classification on average by 4.2%, retrieval by 5.9%, and linear probing by 2.7% over baselines, showing an efficient path toward spectrum-aware vision-language learning for Earth observation. Project page: https://ikhado.github.io/sattxt/

Related papers

• Exploring the Potential of Contrastive Language-Image Pre-training for Multi-Source Remote Sensing Data [12.302498570831538]
  We propose an end-to-end contrastive learning framework that supports multi-source sensor inputs for vision-language modeling.<n>We introduce a spectral-context-aware mask-based contrastive learning scheme to suppress modality-specific redundant features.<n>To support multi-modal training, we construct OmniRS5M, the first large-scale remote sensing image-text corpus.
  arXiv  Detail & Related papers  (2026-09-03T05:45:31Z)
• VEGA: Visual Encoder Grounding Alignment for Spatially-Aware Vision-Language-Action Models [55.12929235609365]
  Visual backbones of current vision-language-action (VLA) models are predominantly pretrained on 2D image data without explicit 3D geometric supervision.<n>Existing implicit grounding methods partially address this by aligning VLA features with those of 3D-aware foundation models.<n>We propose VEGA, a framework that directly aligns the output of the VLA's visual encoder with spatially-aware features from DINOv2-FiT3D.
  arXiv  Detail & Related papers  (2026-05-11T12:44:26Z)
• SpecPL: Disentangling Spectral Granularity for Prompt Learning [11.908475229249161]
  We introduce Disentangling Spectral Granularity for Prompt Learning (SpecPL), which approaches prompt learning from a novel spectral perspective.<n> Specifically, we leverage a frozen VAE to decompose visual signals into semantic low-frequency bands and granular high-frequency details.<n>Experiments on 11 benchmarks demonstrate competitive state-of-the-art performance, achieving a new performance ceiling of 81.51% harmonic-mean accuracy.
  arXiv  Detail & Related papers  (2026-05-06T05:13:40Z)
• SpectraDINO: Bridging the Spectral Gap in Vision Foundation Models via Lightweight Adapters [1.2622634782102324]
  Vision Foundation Models (VFMs) pretrained on large-scale RGB data have demonstrated remarkable representation quality.<n>Their applicability to multispectral imaging spanning Near-Infrared (NIR), Short-Wave Infrared (SWIR), and Long-Wave Infrared (LWIR) remains largely unexplored.<n>We present SpectraDINO, a multispectral VFM that bridges this spectral gap by extending DINOv2 ViT backbones to beyond-visible modalities.
  arXiv  Detail & Related papers  (2026-05-04T06:09:13Z)
• STVG-R1: Incentivizing Instance-Level Reasoning and Grounding in Videos via Reinforcement Learning [65.36458157092207]
  In vision-language models (VLMs), misalignment between textual descriptions and visual coordinates often induces hallucinations.<n>We propose a novel visual prompting paradigm that avoids the difficult problem of aligning coordinates across modalities.<n>We introduce STVG-R1, the first reinforcement learning framework for STVG, which employs a task-driven reward to jointly optimize temporal accuracy, spatial consistency, and structural format regularization.
  arXiv  Detail & Related papers  (2026-02-12T08:53:32Z)
• Self-supervised and Multi-fidelity Learning for Extended Predictive Soil Spectroscopy [2.8830677829565894]
  We propose a framework for multi-fidelity learning and extended predictive soil spectroscopy based on latent space embeddings.<n>A self-supervised representation was pretrained with the large MIR spectral library and the Vari Autoencoder algorithm.<n> Predictions from the spectrum conversion (NIR to MIR) task did not match the performance of the original MIR spectra but were similar or superior to predictive performance of NIR-only models.
  arXiv  Detail & Related papers  (2025-11-20T01:36:33Z)
• S2ML: Spatio-Spectral Mutual Learning for Depth Completion [56.26679539288063]
  Raw depth images captured by RGB-D cameras often suffer from incomplete depth values due to weak reflections, boundary shadows, and artifacts.<n>Existing methods address this problem through depth completion in the image domain, but they overlook the physical characteristics of raw depth images.<n>We propose a Spatio-Spectral Mutual Learning framework (S2ML) to harmonize the advantages of both spatial and frequency domains for depth completion.
  arXiv  Detail & Related papers  (2025-11-08T15:01:55Z)
• Spatial-SSRL: Enhancing Spatial Understanding via Self-Supervised Reinforcement Learning [93.19037653970622]
  We introduce Spatial-SSRL, a self-supervised RL paradigm that derives verifiable signals directly from ordinary RGB or RGB-D images.<n>Training on our tasks substantially improves spatial reasoning while preserving general visual capabilities.<n>Our results show that simple, intrinsic supervision enables RLVR at scale and provides a practical route to stronger spatial intelligence in LVLMs.
  arXiv  Detail & Related papers  (2025-10-31T16:30:08Z)
• Sky Background Building of Multi-objective Fiber spectra Based on Mutual Information Network [13.537469764303637]
  We propose a sky background estimation model: Sky background building based on Mutual Information (SMI)<n>SMI based on mutual information and incremental training approach.<n>Results show that SMI can obtain a better object sky background during the observation, especially in the blue end.
  arXiv  Detail & Related papers  (2025-08-27T13:36:12Z)
• LED: LLM Enhanced Open-Vocabulary Object Detection without Human Curated Data Generation [52.58791563814837]
  Large foundation models trained on large-scale vision-language data can boost Open-Vocabulary Object Detection (OVD)<n>This paper presents a systematic method to enhance visual grounding by utilizing decoder layers of the Large Language Models (LLMs)<n>We find that intermediate LLM layers already encode rich spatial semantics; adapting only the early layers yields most of the gain.
  arXiv  Detail & Related papers  (2025-03-18T00:50:40Z)
• Bringing RGB and IR Together: Hierarchical Multi-Modal Enhancement for Robust Transmission Line Detection [67.02804741856512]
  We propose a novel Hierarchical Multi-Modal Enhancement Network (HMMEN) that integrates RGB and IR data for robust and accurate TL detection.<n>Our method introduces two key components: (1) a Mutual Multi-Modal Enhanced Block (MMEB), which fuses and enhances hierarchical RGB and IR feature maps in a coarse-to-fine manner, and (2) a Feature Alignment Block (FAB) that corrects misalignments between decoder outputs and IR feature maps by leveraging deformable convolutions.
  arXiv  Detail & Related papers  (2025-01-25T06:21:06Z)
• A generic self-supervised learning (SSL) framework for representation learning from spectra-spatial feature of unlabeled remote sensing imagery [4.397725469518669]
  Self-supervised learning (SSL) enables the models to learn a representation from orders of magnitude more unlabelled data. This work has designed a novel SSL framework that is capable of learning representation from both spectra-spatial information of unlabelled data.
  arXiv  Detail & Related papers  (2023-06-27T23:50:43Z)
• L1BSR: Exploiting Detector Overlap for Self-Supervised Single-Image Super-Resolution of Sentinel-2 L1B Imagery [14.233972890133769]
  High-resolution satellite imagery is a key element for many Earth monitoring applications. The lack of reliable high-resolution ground truth limits the application of deep learning methods to this task. We propose L1BSR, a deep learning-based method for single-image super-resolution and band alignment of Sentinel-2 L1B 10m bands.
  arXiv  Detail & Related papers  (2023-04-14T00:17:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.