End-to-End Unmixing with Material Prompts for Hyperspectral Object Tracking

• URL: http://arxiv.org/abs/2605.20569v1
• Date: Wed, 20 May 2026 00:01:05 GMT
• Title: End-to-End Unmixing with Material Prompts for Hyperspectral Object Tracking
• Abstract summary: Hyperspectral imagery encodes rich material properties that can improve tracking robustness under appearance ambiguity, illumination change, and background clutter.<n>We propose a material representation decomposition module for deep learning-based spectral unmixing with adaptive frequency decomposition.<n>We also introduce a dual-branch wavelet-enhanced material prompt module that learns low- and high-frequency material prompts.
• Score: 10.366723514972815
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Hyperspectral imagery encodes rich material properties that can improve tracking robustness under appearance ambiguity, illumination change, and background clutter. However, due to the limited availability of hyperspectral video data, many existing methods adapt pretrained RGB trackers via spatial or channel fusion strategies, largely neglecting the intrinsic material information in hyperspectral imagery. Moreover, the few material-aware approaches typically rely on external spectral unmixing pipelines that are decoupled from the tracking objective, limiting effective optimization of material representations for target localization. To address these limitations, we formulate hyperspectral object tracking as a joint optimization problem of material decomposition and target localization, coupling the two tasks via a weighted target-oriented unmixing loss that explicitly aligns material representations with localization accuracy. Specifically, we propose a material representation decomposition module for deep learning-based spectral unmixing with adaptive frequency decomposition. Building on the decomposed material representations, we further introduce a dual-branch wavelet-enhanced material prompt module that learns low- and high-frequency material prompts through efficient spatial-material interactions in the frequency domain. The framework is model-agnostic and can be seamlessly generalized to different unmixing backbones. Extensive experiments on standard hyperspectral tracking benchmarks demonstrate state-of-the-art performance and validate the effectiveness of the proposed end-to-end material-aware tracking framework. Code is available at https://github.com/han030927/E2EMPT.

Related papers

• MSP-Net: Manifold-Guided Spectral Prompt Network for Hyperspectral Object Tracking [11.03462046191901]
  Hyperspectral object tracking offers unique advantages for target discrimination in complex scenes.<n>Existing methods treat hyperspectral images as multi-channel extensions of RGB images, performing feature fusion in fixed band order.<n>We propose the Manifold-Guided Spectral Prompt Network (MSP-Net)<n> Experiments on HOT2020 and HOT2023 demonstrate MSP-Net achieves AUC and Precision exceeding 0.80 and 0.96, respectively.
  arXiv  Detail & Related papers  (2026-08-10T13:09:25Z)
• Active Adversarial Perturbation-driven Associative Memory Retrieval for RGB-Event Visual Object Tracking [58.338848435042536]
  RGB-Event tracking improves localization by fusing RGB appearance textures and dense temporal motion cues from event sensors.<n>Real-world scenes suffer diverse structured signal degradations that hinder traditional multi-modal fusion.<n>We present a hierarchical perturbation and retrieval framework tailored for RGB-Event tracking with robustness against partial target missing and modal degradation.
  arXiv  Detail & Related papers  (2026-06-24T23:34:06Z)
• Dynamic Spectral Denoising with Global-Context Attention for Multi-Behavior Recommendation [58.33732779555114]
  We argue that the key bottleneck is a representation-level failure caused by two coupled heterogeneities.<n>We propose Dynamic Spectral Denoising with Global-Context Attention for Multi-Behavior Recommendation (SpectraMB)<n>SpectraMB achieves the best results in most evaluation settings and exhibits improved robustness under noisy interactions.
  arXiv  Detail & Related papers  (2026-06-01T15:58:13Z)
• MODA: The First Challenging Benchmark for Multispectral Object Detection in Aerial Images [26.48439423478357]
  We introduce the first large-scale dataset for Multispectral Object Detection in Aerial images (MODA)<n>This dataset comprises 14,041 MSIs and 330,191 annotations across diverse, challenging scenarios.<n>We also propose OSSDet, a framework that integrates spectral and spatial information with object-aware cues.
  arXiv  Detail & Related papers  (2025-12-10T10:07:06Z)
• MaterialRefGS: Reflective Gaussian Splatting with Multi-view Consistent Material Inference [83.38607296779423]
  We show that multi-view consistent material inference with more physically-based environment modeling is key to learning accurate reflections with Gaussian Splatting.<n>Our method faithfully recovers both illumination and geometry, achieving state-of-the-art rendering quality in novel views synthesis.
  arXiv  Detail & Related papers  (2025-10-13T13:29:20Z)
• Hyperspectral Adapter for Semantic Segmentation with Vision Foundation Models [18.24287471339871]
  Hyperspectral imaging (HSI) captures spatial information along with dense spectral measurements across numerous narrow wavelength bands.<n>Our architecture incorporates a spectral transformer and a spectrum-aware spatial prior module to extract rich spatial-spectral features.<n>Our architecture achieves state-of-the-art semantic segmentation performance while directly using HSI inputs, outperforming both vision-based and hyperspectral segmentation methods.
  arXiv  Detail & Related papers  (2025-09-24T13:32:07Z)
• MGDFIS: Multi-scale Global-detail Feature Integration Strategy for Small Object Detection [12.838872442435527]
  Small object detection in UAV imagery is crucial for applications such as search-and-rescue, traffic monitoring, and environmental surveillance.<n>Existing multi-scale fusion methods help, but add computational burden and blur fine details.<n>We propose a unified fusion framework that tightly couples global context with local detail to boost detection performance.
  arXiv  Detail & Related papers  (2025-06-15T02:54:25Z)
• Optimizing Cooperative Multi-Object Tracking using Graph Signal Processing [45.68287260385148]
  This paper proposes a novel Cooperative MOT framework for tracking objects in 3D LiDAR scene.<n>By exploiting a fully connected graph topology defined by the detected bounding boxes, we employ the Graph Laplacian processing optimization technique.<n>An extensive evaluation study has been conducted, using the real-world V2V4Real dataset.
  arXiv  Detail & Related papers  (2025-06-11T07:21:58Z)
• AuxDet: Auxiliary Metadata Matters for Omni-Domain Infrared Small Target Detection [49.81255045696323]
  We present the Auxiliary Metadata Driven Infrared Small Target Detector (AuxDet)<n>AuxDet integrates metadata semantics with visual features, guiding adaptive representation learning for each sample.<n>Experiments on the challenging WideIRSTD-Full benchmark demonstrate that AuxDet consistently outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-05-21T07:02:05Z)
• Frequency Perception Network for Camouflaged Object Detection [51.26386921922031]
  We propose a novel learnable and separable frequency perception mechanism driven by the semantic hierarchy in the frequency domain.<n>Our entire network adopts a two-stage model, including a frequency-guided coarse localization stage and a detail-preserving fine localization stage.<n>Compared with the currently existing models, our proposed method achieves competitive performance in three popular benchmark datasets.
  arXiv  Detail & Related papers  (2023-08-17T11:30:46Z)
• RRNet: Relational Reasoning Network with Parallel Multi-scale Attention for Salient Object Detection in Optical Remote Sensing Images [82.1679766706423]
  Salient object detection (SOD) for optical remote sensing images (RSIs) aims at locating and extracting visually distinctive objects/regions from the optical RSIs. We propose a relational reasoning network with parallel multi-scale attention for SOD in optical RSIs. Our proposed RRNet outperforms the existing state-of-the-art SOD competitors both qualitatively and quantitatively.
  arXiv  Detail & Related papers  (2021-10-27T07:18:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.