SpecPL: Disentangling Spectral Granularity for Prompt Learning

• URL: http://arxiv.org/abs/2605.04504v1
• Date: Wed, 06 May 2026 05:13:40 GMT
• Title: SpecPL: Disentangling Spectral Granularity for Prompt Learning
• Abstract summary: We introduce Disentangling Spectral Granularity for Prompt Learning (SpecPL), which approaches prompt learning from a novel spectral perspective.<n> Specifically, we leverage a frozen VAE to decompose visual signals into semantic low-frequency bands and granular high-frequency details.<n>Experiments on 11 benchmarks demonstrate competitive state-of-the-art performance, achieving a new performance ceiling of 81.51% harmonic-mean accuracy.
• Score: 11.908475229249161
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Existing prompt learning for VLMs exhibits a modality asymmetry, predominantly optimizing text tokens while still relying on frozen visual encoder as holistic extractor and neglecting the spectral granularity essential for fine-grained discrimination. To bridge this, we introduce Disentangling Spectral Granularity for Prompt Learning (SpecPL), which approaches prompt learning from a novel spectral perspective via Counterfactual Granule Supervision. Specifically, we leverage a frozen VAE to decompose visual signals into semantic low-frequency bands and granular high-frequency details. A frozen Visual Semantic Bank anchors text representations to universal low-frequency invariants, mitigating overfitting. Crucially, fine-grained discrimination is driven by counterfactual granule training: by permuting high-frequency signals, we compel the model to explicitly distinguish visual granularity from semantic invariance. Uniquely, SpecPL serves as a universal plug-and-play booster, revitalizing text-oriented baselines like CoOp and MaPLe via visual-side guidance. Experiments on 11 benchmarks demonstrate competitive state-of-the-art performance, achieving a new performance ceiling of 81.51\% harmonic-mean accuracy. These results validate that spectral disentanglement with counterfactual supervision effectively bridges the gap in the stability-generalization trade-off. Code is released at https://github.com/Mlrac1e/SpecPL-Prompt-Learning.

Related papers

• HAFI-VLM: A Frequency Perspective for Diagnosing and Enhancing Visual Perception in Vision-Language Models [9.926121668688772]
  Vision-language models (VLMs) remain unreliable when predictions require fine-grained visual evidence.<n>We identify a previously overlooked cause: spectral response rigidity.<n>We propose HAFI-VLM, which introduces a task-conditioned frequency pathway while preserving the pretrained semantic representation.
  arXiv  Detail & Related papers  (2026-08-03T12:15:33Z)
• Dynamic Spectral Denoising with Global-Context Attention for Multi-Behavior Recommendation [58.33732779555114]
  We argue that the key bottleneck is a representation-level failure caused by two coupled heterogeneities.<n>We propose Dynamic Spectral Denoising with Global-Context Attention for Multi-Behavior Recommendation (SpectraMB)<n>SpectraMB achieves the best results in most evaluation settings and exhibits improved robustness under noisy interactions.
  arXiv  Detail & Related papers  (2026-06-01T15:58:13Z)
• LISA: Language-guided Interference-aware Spatial-Frequency Attention for Driver Gaze Estimation [12.034295331199813]
  Driver gaze estimation serves as a fundamental metric for evaluating driver attentiveness in modern monitoring systems.<n>We propose LISA, a textbfLanguage-guided textbfInterference-aware textbfSpatial-Frequency textbfAttention framework.
  arXiv  Detail & Related papers  (2026-05-17T06:57:31Z)
• Disentangle-then-Refine: LLM-Guided Decoupling and Structure-Aware Refinement for Graph Contrastive Learning [8.080589414732982]
  Conventional Graph Contrastive Learning (GCL) on Text-Attributed Graphs (TAGs) relies on blind augmentations, inadvertently entangling task-relevant signals with noise.<n>We propose SDM-SCR, a robust framework anchored in Approximate Orthogonal Decomposition.
  arXiv  Detail & Related papers  (2026-04-16T07:57:11Z)
• Spectrum Matching: a Unified Perspective for Superior Diffusability in Latent Diffusion [20.731913778080344]
  We show that pixel-space diffusion trained with an MSE objective is inherently biased toward learning low and mid frequencies.<n>Motivated by this result, we propose the emphSpectrum Matching Hypothesis.<n>We show that Spectrum Matching yields superior diffusion generation on CelebA and ImageNet datasets.
  arXiv  Detail & Related papers  (2026-03-15T22:47:43Z)
• The Prism Hypothesis: Harmonizing Semantic and Pixel Representations via Unified Autoencoding [82.53463660564933]
  semantic encoders primarily capture low-frequency components that encode abstract meaning, whereas pixel encoders retain high-frequency information that conveys fine-grained detail.<n>We propose Unified Autoencoding (UAE), a model that harmonizes semantic structure and pixel details via an innovative frequency-band modulator.
  arXiv  Detail & Related papers  (2025-12-22T18:59:57Z)
• Diversity Boosts AI-Generated Text Detection [51.56484100374058]
  DivEye is a novel framework that captures how unpredictability fluctuates across a text using surprisal-based features.<n>Our method outperforms existing zero-shot detectors by up to 33.2% and achieves competitive performance with fine-tuned baselines.
  arXiv  Detail & Related papers  (2025-09-23T10:21:22Z)
• Generalize across Homophily and Heterophily: Hybrid Spectral Graph Pre-Training and Prompt Tuning [32.2698194084332]
  Existing methods fail to handle diverse spectral distributions in real-world graphs with varying homophily.<n>We propose a novel framework that ensures spectral alignment throughout both pre-training and prompt-tuning.<n>Our experiments validate the effectiveness under both transductive and inductive learning settings.
  arXiv  Detail & Related papers  (2025-08-15T08:55:57Z)
• SpecBPP: A Self-Supervised Learning Approach for Hyperspectral Representation and Soil Organic Carbon Estimation [9.292155894591877]
  We propose a novel self-supervised learning framework for hyperspectral imagery.<n>SpecBPP challenges a model to recover the correct order of shuffled spectral segments.<n>Our results demonstrate that spectral order prediction is a powerful pretext task for hyperspectral understanding.
  arXiv  Detail & Related papers  (2025-07-26T04:11:43Z)
• ICPL-ReID: Identity-Conditional Prompt Learning for Multi-Spectral Object Re-Identification [25.953780086825457]
  Multi-spectral object re-identification (ReID) brings a new perception perspective for smart city and intelligent transportation applications.<n>Most existing methods fuse spectral data through intricate modal interaction modules, lacking fine-grained semantic understanding of spectral information.<n>We propose a novel Identity-Conditional text Prompt Learning framework (ICPL), which exploits the powerful cross-modal alignment capability of CLIP.
  arXiv  Detail & Related papers  (2025-05-23T12:38:27Z)
• Empowering Sparse-Input Neural Radiance Fields with Dual-Level Semantic Guidance from Dense Novel Views [66.1245505423179]
  We show that rendered semantics can be treated as a more robust form of augmented data than rendered RGB.<n>Our method enhances NeRF's performance by incorporating guidance derived from the rendered semantics.
  arXiv  Detail & Related papers  (2025-03-04T03:13:44Z)
• KNN Transformer with Pyramid Prompts for Few-Shot Learning [52.735070934075736]
  Few-Shot Learning aims to recognize new classes with limited labeled data. Recent studies have attempted to address the challenge of rare samples with textual prompts to modulate visual features.
  arXiv  Detail & Related papers  (2024-10-14T07:39:30Z)
• Scalable Representation Learning in Linear Contextual Bandits with Constant Regret Guarantees [103.69464492445779]
  We propose BanditSRL, a representation learning algorithm that learns a realizable representation with good spectral properties. We prove that BanditSRL can be paired with any no-regret algorithm and achieve constant regret whenever an HLS representation is available.
  arXiv  Detail & Related papers  (2022-10-24T10:04:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.