Feature Visualization Recovers Known Cortical Selectivity from TRIBE v2

• URL: http://arxiv.org/abs/2605.13904v1
• Date: Wed, 13 May 2026 00:55:45 GMT
• Title: Feature Visualization Recovers Known Cortical Selectivity from TRIBE v2
• Abstract summary: Brain encoder models predict cortical fMRI responses from the internal activations of pretrained vision and language networks.<n>We propose feature visualization as a complementary interpretability technique.<n>The probe is simple, differentiable, and applicable to any brain encoder with a differentiable backbone.
• Score: 0.19336815376402716
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Brain encoder models predict cortical fMRI responses from the internal activations of pretrained vision and language networks, and are typically evaluated by held-out prediction accuracy. This is a useful signal for training but a poor one for interpretation: it tells us an encoder fits the data without telling us whether it has internalized the functional organization of the brain. We propose feature visualization -- gradient ascent on the encoder's predicted activation for a target region of interest (ROI) -- as a complementary interpretability technique, and apply it to TRIBE v2 composed with V-JEPA 2 (ViT-G, 40 layers), holding both frozen and synthesizing still images for seven regions spanning the ventral and dorsal visual hierarchies. Under identical hyperparameters, the probe recovers a visible progression of increasing spatial scale and feature complexity across V1 to V4, matching the ventral-stream hierarchy. It also produces three distinctive downstream regimes: radial "frozen-motion" streaks for the middle temporal area (MT) despite static-only optimization, face-like features for the fusiform face area (FFA), and consistent rectilinear line patterns for the parahippocampal place area (PPA). Optimized FFA stimuli drive the predicted region ~4x as much as a natural face photograph, consistent with feature visualization producing adversarial super-stimuli rather than canonical exemplars. The probe is simple, differentiable, and applicable to any brain encoder with a differentiable backbone, allowing for qualitative evaluation of brain encoders.

Related papers

• ProgFormer: Hierarchical Voxel Diffusion Transformer for Longitudinal Brain MRI Prediction [81.07048249760953]
  longitudinal changes are often subtle and confined to specific anatomical regions.<n>Most subject-specific brain structure remains stable over time.<n>ProgFormer is a hierarchical voxel-space Diffusion Transformer for longitudinal brain MRI prediction.
  arXiv  Detail & Related papers  (2026-07-30T00:13:42Z)
• Coarse-to-fine Hierarchical Architecture with Sequential Mamba for Brain Reconstruction [5.732406989501624]
  We propose CHASMBrain, a novel hierarchical two-stage framework for image-to-fMRI encoding.<n>Our architecture leverages a dual-stream Mamba design to explicitly separate and process global semantic tokens and local spatial patches.<n>Our method achieves a Pearson correlation of 0.429 and an MSE of 0.261, outperforming all evaluated baselines.
  arXiv  Detail & Related papers  (2026-06-03T11:53:23Z)
• UniV2D: Bridging Visual Restoration and Semantic Perception for Underwater Salient Object Detection [71.83097731030254]
  We propose a Unified Vision-to-Detection Network (UniV2D) to jointly optimize visual restoration and salient object detection.<n>UniV2D introduces a semantic-driven learning paradigm: high-level saliency semantics actively guide the restoration process.<n>It significantly outperforms state-of-the-art methods in both quantitative and qualitative evaluations.
  arXiv  Detail & Related papers  (2026-05-08T02:31:45Z)
• Interpreting V1 Population Activity via Image-Neural Latent Representation Alignment [8.749824371158454]
  Recent alignment based approaches have improved the accuracy of decoding visual stimuli from brain activity.<n>We propose Dual-Tower Image-Neural Alignment (DINA), an interpretable contrastive framework for analyzing population level visual computations in primary visual cortex (V1)<n>DINA jointly trains a biologically motivated dual-tower architecture that aligns visual stimuli and corresponding V1 population responses in a shared latent space at the level of intermediate feature maps.
  arXiv  Detail & Related papers  (2026-05-05T21:15:41Z)
• HyFI: Hyperbolic Feature Interpolation for Brain-Vision Alignment [12.235537797397798]
  We propose a novel framework, Hyperbolic Feature Interpolation (HyFI), which interpolates between semantic and perceptual visual features along hyperbolic geodesics.<n>HyFI achieves state-of-the-art performance in zero-shot brain-to-image retrieval, outperforming prior methods with Top-1 accuracy improvements of up to +17.3%.
  arXiv  Detail & Related papers  (2026-03-24T02:35:54Z)
• Hi-DREAM: Brain Inspired Hierarchical Diffusion for fMRI Reconstruction via ROI Encoder and visuAl Mapping [5.019958634393433]
  Hi-DREAM is a conditional diffusion framework that makes the cortical organization explicit.<n>A lightweight, depth-matched ControlNet injects scale-specific hints during denoising.<n>Experiments show that Hi-DREAM attains state-of-the-art performance on high-level semantic metrics.
  arXiv  Detail & Related papers  (2025-11-14T16:05:44Z)
• Subsampled Randomized Fourier GaLore for Adapting Foundation Models in Depth-Driven Liver Landmark Segmentation [6.91206648866302]
  We propose a depth-guided liver landmark segmentation framework integrating semantic and geometric cues via vision foundation encoders.<n>To efficiently adapt SAM2, we introduce SRFT-GaLore, a novel low-rank gradient projection method that replaces the computationally expensive SVD with a Subsampled Randomized Fourier Transform.<n>Our method achieves a 4.85% improvement in Dice Similarity Coefficient and a 11.78-point reduction in Average Symmetric Surface Distance compared to the D2GPLand.
  arXiv  Detail & Related papers  (2025-11-05T04:16:49Z)
• Sparse Autoencoders Bridge The Deep Learning Model and The Brain [18.058358411706052]
  We present SAE-BrainMap, a novel framework that aligns deep learning visual model representations with voxel-level fMRI responses.<n>It is found that ViT-B/16$_CLIP$ tends to utilize low-level information to generate high-level semantic information in the early layers.<n>Our results establish a direct, downstream-task-free bridge between deep neural networks and human visual cortex, offering new insights into model interpretability.
  arXiv  Detail & Related papers  (2025-06-10T06:35:14Z)
• Empowering Sparse-Input Neural Radiance Fields with Dual-Level Semantic Guidance from Dense Novel Views [66.1245505423179]
  We show that rendered semantics can be treated as a more robust form of augmented data than rendered RGB.<n>Our method enhances NeRF's performance by incorporating guidance derived from the rendered semantics.
  arXiv  Detail & Related papers  (2025-03-04T03:13:44Z)
• It Takes Two: Accurate Gait Recognition in the Wild via Cross-granularity Alignment [72.75844404617959]
  This paper proposes a novel cross-granularity alignment gait recognition method, named XGait. To achieve this goal, the XGait first contains two branches of backbone encoders to map the silhouette sequences and the parsing sequences into two latent spaces. Comprehensive experiments on two large-scale gait datasets show XGait with the Rank-1 accuracy of 80.5% on Gait3D and 88.3% CCPG.
  arXiv  Detail & Related papers  (2024-11-16T08:54:27Z)
• KFD-NeRF: Rethinking Dynamic NeRF with Kalman Filter [49.85369344101118]
  We introduce KFD-NeRF, a novel dynamic neural radiance field integrated with an efficient and high-quality motion reconstruction framework based on Kalman filtering. Our key idea is to model the dynamic radiance field as a dynamic system whose temporally varying states are estimated based on two sources of knowledge: observations and predictions. Our KFD-NeRF demonstrates similar or even superior performance within comparable computational time and state-of-the-art view synthesis performance with thorough training.
  arXiv  Detail & Related papers  (2024-07-18T05:48:24Z)
• S^2Former-OR: Single-Stage Bi-Modal Transformer for Scene Graph Generation in OR [50.435592120607815]
  Scene graph generation (SGG) of surgical procedures is crucial in enhancing holistically cognitive intelligence in the operating room (OR) Previous works have primarily relied on multi-stage learning, where the generated semantic scene graphs depend on intermediate processes with pose estimation and object detection. In this study, we introduce a novel single-stage bi-modal transformer framework for SGG in the OR, termed S2Former-OR.
  arXiv  Detail & Related papers  (2024-02-22T11:40:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.