Beyond Compression: Quantifying Spectral Accessibility in Vision Representations

• URL: http://arxiv.org/abs/2606.03795v1
• Date: Tue, 02 Jun 2026 15:42:54 GMT
• Title: Beyond Compression: Quantifying Spectral Accessibility in Vision Representations
• Abstract summary: Vision-language models map visual features into a shared embedding space through learned projection layers.<n>This study examines changes in representation through spatial-frequency accessibility, measured by the linear recoverability of band-limited energy from model representations.
• Score: 4.681075707831016
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-language models map visual features into a shared embedding space through learned projection layers, yet it remains unclear how these transformations alter the structure of visual information. This study examines changes in representation through spatial-frequency accessibility, measured by the linear recoverability of band-limited Fourier energy from model representations. To isolate effects beyond dimensionality reduction, we introduce Residual Spectral Loss (RSL), which evaluates changes relative to a dimension-matched random projection baseline. To reduce confounding effects from optimization, the analysis uses pretrained models with all parameters frozen. The experimental results show consistent frequency-dependent changes in accessibility across CLIP and DINOv2 on ImageNet and MS-COCO datasets. Spectral accessibility follows a non-monotonic trajectory across depth, peaking at intermediate layers before decreasing toward the output representation. The final transformation differs across architectures: CLIP's learned projection is spectrally neutral, with changes explained by compression, whereas DINOv2's [CLS] pooling induces a structured loss across the spectrum. These findings identify intermediate layers and pooling mechanisms as primary drivers of spectral transformation in modern vision encoders.

Related papers

• Implicit Neural Representations: A Signal Processing Perspective [54.57279006229212]
  Implicit neural representations (INRs) mark a fundamental shift in signal modeling, moving from discrete sampled data to continuous functional representations.<n>This article examines the evolution of INRs from a signal processing perspective, emphasizing spectral behavior, sampling theory, and multiscale representation.<n>We highlight the utility of INRs across a broad range of applications, including inverse problems in medical and radar imaging, compression, and 3D scene representation.
  arXiv  Detail & Related papers  (2026-04-16T14:12:06Z)
• SIEFormer: Spectral-Interpretable and -Enhanced Transformer for Generalized Category Discovery [14.288193104482986]
  SIEFormer is composed of two main branches, each corresponding to an implicit and explicit spectral perspective of the ViT.<n>The implicit branch realizes the use of different types of graph Laplacians to model the local structure correlations of tokens.<n>The explicit branch, on the other hand, introduces a Maneuverable Filtering Layer (MFL) that learns global dependencies among tokens.
  arXiv  Detail & Related papers  (2026-02-13T16:22:31Z)
• Suppressing VLM Hallucinations with Spectral Representation Filtering [49.52703800684483]
  Vision-language models (VLMs) frequently produce hallucinations in the form of descriptions of objects, attributes, or relations that do not exist in the image.<n>We introduce Spectral Representation Filtering (SRF), a lightweight, training-free method to suppress such hallucinations by analyzing and correcting the covariance structure of the model's representations.
  arXiv  Detail & Related papers  (2025-11-15T13:49:27Z)
• Fourier-Invertible Neural Encoder (FINE) for Homogeneous Flows [4.095418032380801]
  Invertible neural networks have attracted attention for their compactness, interpretability, and information-preserving properties.<n>We propose the Fourier-Invertible Neural (FINE), which combines invertible monotonic activation functions with reversible filter structures.
  arXiv  Detail & Related papers  (2025-05-21T10:02:59Z)
• SinBasis Networks: Matrix-Equivalent Feature Extraction for Wave-Like Optical Spectrograms [8.37266944852829]
  We propose a unified, matrix-equivalent framework that reinterprets convolution and attention as linear transforms on flattened inputs.<n> Embedding these transforms into CNN, ViT and Capsule architectures yields Sin-Basis Networks with heightened sensitivity to periodic motifs.
  arXiv  Detail & Related papers  (2025-05-06T16:16:42Z)
• AS-GCL: Asymmetric Spectral Augmentation on Graph Contrastive Learning [25.07818336162072]
  Graph Contrastive Learning (GCL) has emerged as the foremost approach for self-supervised learning on graph-structured data.<n>We propose a novel paradigm called AS-GCL that incorporates asymmetric spectral augmentation for graph contrastive learning.<n>Our method introduces significant enhancements to each of these components.
  arXiv  Detail & Related papers  (2025-02-19T08:22:57Z)
• Point Cloud Denoising With Fine-Granularity Dynamic Graph Convolutional Networks [58.050130177241186]
  Noise perturbations often corrupt 3-D point clouds, hindering downstream tasks such as surface reconstruction, rendering, and further processing. This paper introduces finegranularity dynamic graph convolutional networks called GDGCN, a novel approach to denoising in 3-D point clouds.
  arXiv  Detail & Related papers  (2024-11-21T14:19:32Z)
• HoloNets: Spectral Convolutions do extend to Directed Graphs [59.851175771106625]
  Conventional wisdom dictates that spectral convolutional networks may only be deployed on undirected graphs. Here we show this traditional reliance on the graph Fourier transform to be superfluous. We provide a frequency-response interpretation of newly developed filters, investigate the influence of the basis used to express filters and discuss the interplay with characteristic operators on which networks are based.
  arXiv  Detail & Related papers  (2023-10-03T17:42:09Z)
• ESSAformer: Efficient Transformer for Hyperspectral Image Super-resolution [76.7408734079706]
  Single hyperspectral image super-resolution (single-HSI-SR) aims to restore a high-resolution hyperspectral image from a low-resolution observation. We propose ESSAformer, an ESSA attention-embedded Transformer network for single-HSI-SR with an iterative refining structure.
  arXiv  Detail & Related papers  (2023-07-26T07:45:14Z)
• Exploring Invariant Representation for Visible-Infrared Person Re-Identification [77.06940947765406]
  Cross-spectral person re-identification, which aims to associate identities to pedestrians across different spectra, faces a main challenge of the modality discrepancy. In this paper, we address the problem from both image-level and feature-level in an end-to-end hybrid learning framework named robust feature mining network (RFM) Experiment results on two standard cross-spectral person re-identification datasets, RegDB and SYSU-MM01, have demonstrated state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-02-02T05:24:50Z)
• A Hierarchical Coding Scheme for Glasses-free 3D Displays Based on Scalable Hybrid Layered Representation of Real-World Light Fields [0.6091702876917279]
  Scheme learns stacked multiplicative layers from subsets of light field views determined from different scanning orders. The spatial correlation in layer patterns is exploited with varying low ranks in factorization derived from singular value decomposition on a Krylov subspace. encoding with HEVC efficiently removes intra-view and inter-view correlation in low-rank approximated layers.
  arXiv  Detail & Related papers  (2021-04-19T15:09:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.