Convergence of Spectral Principal Paths: How Deep Networks Distill Linear Representations from Noisy Inputs

• URL: http://arxiv.org/abs/2506.08543v1
• Date: Tue, 10 Jun 2025 08:08:52 GMT
• Title: Convergence of Spectral Principal Paths: How Deep Networks Distill Linear Representations from Noisy Inputs
• Abstract summary: High-level representations have become a central focus in enhancing AI transparency and control, shifting attention from individual neurons or circuits to structured semantic directions that align with human-interpretable concepts.<n>Motivated by the Linear Representation Hypothesis (LRH), we propose the Input-Space Linearity Hypothesis (ISLH), which posits that concept-aligned directions originate in the input space and are selectively amplified with increasing depth.<n>We then introduce the Spectral Principal Path (SPP) framework, which formalizes how deep networks progressively distill linear representations along a small set of dominant spectral directions.
• Score: 17.987141330832582
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: High-level representations have become a central focus in enhancing AI transparency and control, shifting attention from individual neurons or circuits to structured semantic directions that align with human-interpretable concepts. Motivated by the Linear Representation Hypothesis (LRH), we propose the Input-Space Linearity Hypothesis (ISLH), which posits that concept-aligned directions originate in the input space and are selectively amplified with increasing depth. We then introduce the Spectral Principal Path (SPP) framework, which formalizes how deep networks progressively distill linear representations along a small set of dominant spectral directions. Building on this framework, we further demonstrate the multimodal robustness of these representations in Vision-Language Models (VLMs). By bridging theoretical insights with empirical validation, this work advances a structured theory of representation formation in deep networks, paving the way for improving AI robustness, fairness, and transparency.

Related papers

• Vision Pretraining for Dense Spatial Perception [69.54043654361772]
  We study vision pretraining through a boundary-centric lens, motivated by the premise that boundaries and shape discontinuities offer essential cues for perceiving geometric properties.<n>Our findings reveal that boundary modeling goes beyond simple line segments and instead serves as a scalable pretraining principle for learning spatially structured visual representations.
  arXiv  Detail & Related papers  (2026-07-06T15:56:14Z)
• Beyond Point-Wise Matching: Structural Representation Alignment for Accelerating Diffusion Transformers [93.3976834364707]
  We propose sREPA, a structural REPresentation Alignment framework to enforce consistency in the relational geometry of feature maps.<n>By encouraging the model to internalize holistic spatial layouts and structural correlations from pre-trained features, sREPA achieves faster and more stable convergence.
  arXiv  Detail & Related papers  (2026-05-16T12:01:04Z)
• Rethinking Token-Level Policy Optimization for Multimodal Chain-of-Thought [73.39221516441624]
  Multimodal Chain-of-Thought (CoT) reasoning requires large vision-language models to construct reasoning trajectories.<n>Existing Reinforcement Learning with Verifiable Rewards (RLVR) methods treat CoT uniformly without distinguishing varying degrees of visual grounding.<n>We propose Perception-Exploration Policy Optimization (PEPO), which derives a perception prior to hidden state similarity and integrates it with token entropy.
  arXiv  Detail & Related papers  (2026-03-24T06:38:00Z)
• Perception-Aware Multimodal Spatial Reasoning from Monocular Images [57.42071289037214]
  spatial reasoning from monocular images is essential for autonomous driving.<n>Current Vision-Language Models (VLMs) still struggle with fine-grained geometric perception.<n>We propose a perception-aware multimodal reasoning framework that equipsVLMs with explicit object-centric grounding ability.
  arXiv  Detail & Related papers  (2026-03-07T02:05:12Z)
• Emergent Structured Representations Support Flexible In-Context Inference in Large Language Models [77.98801218316505]
  Large language models (LLMs) exhibit emergent behaviors suggestive of human-like reasoning.<n>We investigate the internal processing of LLMs during in-context concept inference.
  arXiv  Detail & Related papers  (2026-02-08T03:14:39Z)
• Beyond Pixels: Visual Metaphor Transfer via Schema-Driven Agentic Reasoning [56.24016465596292]
  A visual metaphor constitutes a high-order form of human creativity, employing cross-domain semantic fusion to transform abstract concepts into impactful visual rhetoric.<n>We introduce the task of Visual Metaphor Transfer (VMT), which challenges models to autonomously decouple the "creative essence" from a reference image and re-materialize that abstract logic onto a user-specified subject.<n>Our method significantly outperforms SOTA baselines in metaphor consistency, analogy appropriateness, and visual creativity, paving the way for automated high-impact creative applications in advertising and media.
  arXiv  Detail & Related papers  (2026-02-01T17:01:36Z)
• Multi-step Predictive Coding Leads To Simplicity Bias [2.8933605229876664]
  We investigate how the prediction horizon and network depth shape the solutions of predictive coding tasks.<n>We show that sufficiently deep networks trained with multi-step prediction horizons consistently recover the underlying latent structure.<n>Our results provide a principled understanding of when and why predictive coding induces structured representations.
  arXiv  Detail & Related papers  (2025-11-12T12:57:19Z)
• From Perception to Cognition: A Survey of Vision-Language Interactive Reasoning in Multimodal Large Language Models [66.36007274540113]
  Multimodal Large Language Models (MLLMs) strive to achieve a profound, human-like understanding of and interaction with the physical world.<n>They often exhibit a shallow and incoherent integration when acquiring information (Perception) and conducting reasoning (Cognition)<n>This survey introduces a novel and unified analytical framework: From Perception to Cognition"
  arXiv  Detail & Related papers  (2025-09-29T18:25:40Z)
• Reinforcing Spatial Reasoning in Vision-Language Models with Interwoven Thinking and Visual Drawing [62.447497430479174]
  Drawing to reason in space is a novel paradigm that enables LVLMs to reason through elementary drawing operations in the visual space.<n>Our model, named VILASR, consistently outperforms existing methods across diverse spatial reasoning benchmarks.
  arXiv  Detail & Related papers  (2025-06-11T17:41:50Z)
• Why Representation Engineering Works: A Theoretical and Empirical Study in Vision-Language Models [17.987141330832582]
  We develop a theoretical framework that explains the stability of neural activity across layers using the principal eigenvector.<n>This work transforms Representation Engineering (RepE) into a structured theoretical framework, opening new directions for improving AI robustness, fairness, and transparency.
  arXiv  Detail & Related papers  (2025-03-25T20:32:15Z)
• ASCENT-ViT: Attention-based Scale-aware Concept Learning Framework for Enhanced Alignment in Vision Transformers [29.932706137805713]
  ASCENT-ViT is an attention-based, concept learning framework for Vision Transformers (ViTs)<n>It composes scale and position-aware representations from multiscale feature pyramids and ViT patch representations, respectively.<n>It can be utilized as a classification head on top of standard ViT backbones for improved predictive performance and accurate and robust concept explanations.
  arXiv  Detail & Related papers  (2025-01-16T00:45:05Z)
• Bi-ICE: An Inner Interpretable Framework for Image Classification via Bi-directional Interactions between Concept and Input Embeddings [22.31721412650966]
  Inner interpretability is a promising field focused on uncovering the internal mechanisms of AI systems.<n>We first present a conceptual framework that supports inner interpretability and multilevel analysis for large-scale image classification tasks.<n>We introduce the Bi-directional Interaction between Concept and Input Embeddings (Bi-ICE) module, which facilitates interpretability across the computational, algorithmic, and implementation levels.
  arXiv  Detail & Related papers  (2024-11-26T19:52:00Z)
• Approaching Deep Learning through the Spectral Dynamics of Weights [41.948042468042374]
  spectral dynamics of weights -- the behavior of singular values and vectors during optimization -- to clarify and unify several phenomena in deep learning. We identify a consistent bias in optimization across various experiments, from small-scale grokking'' to large-scale tasks like image classification with ConvNets, image generation with UNets, speech recognition with LSTMs, and language modeling with Transformers.
  arXiv  Detail & Related papers  (2024-08-21T17:48:01Z)
• Learned Image Reasoning Prior Penetrates Deep Unfolding Network for Panchromatic and Multi-Spectral Image Fusion [45.28120834593148]
  We propose a novel model-driven deep unfolding framework with image reasoning prior tailored for the pan-sharpening task. Our framework is motivated by the content reasoning ability of masked autoencoders with insightful designs. The uniqueness of our framework is that the holistic learning process is explicitly integrated with the inherent physical mechanism underlying the pan-sharpening task.
  arXiv  Detail & Related papers  (2023-08-30T15:15:31Z)
• Unsupervised Interpretable Basis Extraction for Concept-Based Visual Explanations [53.973055975918655]
  We show that, intermediate layer representations become more interpretable when transformed to the bases extracted with our method. We compare the bases extracted with our method with the bases derived with a supervised approach and find that, in one aspect, the proposed unsupervised approach has a strength that constitutes a limitation of the supervised one and give potential directions for future research.
  arXiv  Detail & Related papers  (2023-03-19T00:37:19Z)
• Guiding the PLMs with Semantic Anchors as Intermediate Supervision: Towards Interpretable Semantic Parsing [57.11806632758607]
  We propose to incorporate the current pretrained language models with a hierarchical decoder network. By taking the first-principle structures as the semantic anchors, we propose two novel intermediate supervision tasks. We conduct intensive experiments on several semantic parsing benchmarks and demonstrate that our approach can consistently outperform the baselines.
  arXiv  Detail & Related papers  (2022-10-04T07:27:29Z)
• USegScene: Unsupervised Learning of Depth, Optical Flow and Ego-Motion with Semantic Guidance and Coupled Networks [31.600708674008384]
  USegScene is a framework for semantically guided unsupervised learning of depth, optical flow and ego-motion estimation for stereo camera images. We present results on the popular KITTI dataset and show that our approach outperforms other methods by a large margin.
  arXiv  Detail & Related papers  (2022-07-15T13:25:47Z)
• Discovering "Semantics" in Super-Resolution Networks [54.45509260681529]
  Super-resolution (SR) is a fundamental and representative task of low-level vision area. It is generally thought that the features extracted from the SR network have no specific semantic information. Can we find any "semantics" in SR networks?
  arXiv  Detail & Related papers  (2021-08-01T09:12:44Z)
• The Principles of Deep Learning Theory [19.33681537640272]
  This book develops an effective theory approach to understanding deep neural networks of practical relevance. We explain how these effectively-deep networks learn nontrivial representations from training. We show that the depth-to-width ratio governs the effective model complexity of the ensemble of trained networks.
  arXiv  Detail & Related papers  (2021-06-18T15:00:00Z)
• Variational Structured Attention Networks for Deep Visual Representation Learning [49.80498066480928]
  We propose a unified deep framework to jointly learn both spatial attention maps and channel attention in a principled manner. Specifically, we integrate the estimation and the interaction of the attentions within a probabilistic representation learning framework. We implement the inference rules within the neural network, thus allowing for end-to-end learning of the probabilistic and the CNN front-end parameters.
  arXiv  Detail & Related papers  (2021-03-05T07:37:24Z)
• Closed-Form Factorization of Latent Semantics in GANs [65.42778970898534]
  A rich set of interpretable dimensions has been shown to emerge in the latent space of the Generative Adversarial Networks (GANs) trained for synthesizing images. In this work, we examine the internal representation learned by GANs to reveal the underlying variation factors in an unsupervised manner. We propose a closed-form factorization algorithm for latent semantic discovery by directly decomposing the pre-trained weights.
  arXiv  Detail & Related papers  (2020-07-13T18:05:36Z)
• Dynamic Hierarchical Mimicking Towards Consistent Optimization Objectives [73.15276998621582]
  We propose a generic feature learning mechanism to advance CNN training with enhanced generalization ability. Partially inspired by DSN, we fork delicately designed side branches from the intermediate layers of a given neural network. Experiments on both category and instance recognition tasks demonstrate the substantial improvements of our proposed method.
  arXiv  Detail & Related papers  (2020-03-24T09:56:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.