Sparse components distinguish visual pathways & their alignment to neural networks

• URL: http://arxiv.org/abs/2510.08858v1
• Date: Thu, 09 Oct 2025 23:26:11 GMT
• Title: Sparse components distinguish visual pathways & their alignment to neural networks
• Authors: Ammar I Marvi, Nancy G Kanwisher, Meenakshi Khosla,
• Abstract summary: The ventral, dorsal, and lateral streams in high-level human visual cortex are implicated in distinct functional processes.<n>Yet, deep neural networks (DNNs) trained on a single task model the entire visual system surprisingly well.<n>We apply a novel sparse decomposition approach to identify the dominant components of visual representations within each stream.
• Score: 3.466510324781552
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The ventral, dorsal, and lateral streams in high-level human visual cortex are implicated in distinct functional processes. Yet, deep neural networks (DNNs) trained on a single task model the entire visual system surprisingly well, hinting at common computational principles across these pathways. To explore this inconsistency, we applied a novel sparse decomposition approach to identify the dominant components of visual representations within each stream. Consistent with traditional neuroscience research, we find a clear difference in component response profiles across the three visual streams -- identifying components selective for faces, places, bodies, text, and food in the ventral stream; social interactions, implied motion, and hand actions in the lateral stream; and some less interpretable components in the dorsal stream. Building on this, we introduce Sparse Component Alignment (SCA), a new method for measuring representational alignment between brains and machines that better captures the latent neural tuning of these two visual systems. Using SCA, we find that standard visual DNNs are more aligned with the ventral than either dorsal or lateral representations. SCA reveals these distinctions with greater resolution than conventional population-level geometry, offering a measure of representational alignment that is sensitive to a system's underlying axes of neural tuning.

Related papers

• Uncovering Semantic Selectivity of Latent Groups in Higher Visual Cortex with Mutual Information-Guided Diffusion [19.983291706164923]
  We present MIG-Vis, a method to visualize and validate the visual-semantic attributes encoded in neural latent subspaces.<n>We validate MIG-Vis on multi-session neural spiking datasets from the inferior temporal (IT) cortex of two macaques.
  arXiv  Detail & Related papers  (2025-10-02T16:33:40Z)
• Convergent transformations of visual representation in brains and models [0.0]
  A fundamental question in cognitive neuroscience is what shapes visual perception: the external world's structure or the brain's internal architecture.<n>We show a convergent computational solution for visual encoding in both human and artificial vision, driven by the structure of the external world.
  arXiv  Detail & Related papers  (2025-07-18T14:13:54Z)
• Intraoperative Registration by Cross-Modal Inverse Neural Rendering [61.687068931599846]
  We present a novel approach for 3D/2D intraoperative registration during neurosurgery via cross-modal inverse neural rendering. Our approach separates implicit neural representation into two components, handling anatomical structure preoperatively and appearance intraoperatively. We tested our method on retrospective patients' data from clinical cases, showing that our method outperforms state-of-the-art while meeting current clinical standards for registration.
  arXiv  Detail & Related papers  (2024-09-18T13:40:59Z)
• Finding Shared Decodable Concepts and their Negations in the Brain [4.111712524255376]
  We train a highly accurate contrastive model that maps brain responses during naturalistic image viewing to CLIP embeddings. We then use a novel adaptation of the DBSCAN clustering algorithm to cluster the parameters of participant-specific contrastive models. Examining the images most and least associated with each SDC cluster gives us additional insight into the semantic properties of each SDC.
  arXiv  Detail & Related papers  (2024-05-27T21:28:26Z)
• Graph Neural Networks for Learning Equivariant Representations of Neural Networks [55.04145324152541]
  We propose to represent neural networks as computational graphs of parameters. Our approach enables a single model to encode neural computational graphs with diverse architectures. We showcase the effectiveness of our method on a wide range of tasks, including classification and editing of implicit neural representations.
  arXiv  Detail & Related papers  (2024-03-18T18:01:01Z)
• Learning Multimodal Volumetric Features for Large-Scale Neuron Tracing [72.45257414889478]
  We aim to reduce human workload by predicting connectivity between over-segmented neuron pieces. We first construct a dataset, named FlyTracing, that contains millions of pairwise connections of segments expanding the whole fly brain. We propose a novel connectivity-aware contrastive learning method to generate dense volumetric EM image embedding.
  arXiv  Detail & Related papers  (2024-01-05T19:45:12Z)
• A Dual-Stream Neural Network Explains the Functional Segregation of Dorsal and Ventral Visual Pathways in Human Brains [8.24969449883056]
  We develop a dual-stream vision model inspired by the human eyes and brain. At the input level, the model samples two complementary visual patterns. At the backend, the model processes the separate input patterns through two branches of convolutional neural networks.
  arXiv  Detail & Related papers  (2023-10-20T22:47:40Z)
• Prune and distill: similar reformatting of image information along rat visual cortex and deep neural networks [61.60177890353585]
  Deep convolutional neural networks (CNNs) have been shown to provide excellent models for its functional analogue in the brain, the ventral stream in visual cortex. Here we consider some prominent statistical patterns that are known to exist in the internal representations of either CNNs or the visual cortex. We show that CNNs and visual cortex share a similarly tight relationship between dimensionality expansion/reduction of object representations and reformatting of image information.
  arXiv  Detail & Related papers  (2022-05-27T08:06:40Z)
• FuNNscope: Visual microscope for interactively exploring the loss landscape of fully connected neural networks [77.34726150561087]
  We show how to explore high-dimensional landscape characteristics of neural networks. We generalize observations on small neural networks to more complex systems. An interactive dashboard opens up a number of possible application networks.
  arXiv  Detail & Related papers  (2022-04-09T16:41:53Z)
• Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
  We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs. By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
  arXiv  Detail & Related papers  (2022-02-01T17:11:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.