Perturbation-based Regional Interpretability through Subtraction Mapping (PRISM): naming-error dissociations in language models and post-stroke aphasia

• URL: http://arxiv.org/abs/2608.12717v1
• Date: Thu, 13 Aug 2026 02:02:05 GMT
• Title: Perturbation-based Regional Interpretability through Subtraction Mapping (PRISM): naming-error dissociations in language models and post-stroke aphasia
• Abstract summary: Subtraction analysis is the standard framework of human neuroimaging.<n>We develop PRISM (Perturbation-based Regional Interpretability through Subtraction Mapping)<n>PRISM gives a falsifiable, spatially resolved test of functional-specialization claims in transformer language models.
• Score: 6.867794233852404
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Mechanistic interpretability of large language models lacks spatially resolved, falsifiable tools for testing whether internal components are specialized for distinct cognitive operations. We adapt subtraction analysis, the standard framework of human neuroimaging, from biological brains to perturbed transformers, and apply the same logic to both substrates in parallel. Building on the Brain-LLM Unified Model (BLUM), which showed that layer-perturbed LLaVA-1.6-Vicuna-13B error profiles match the lesion patterns of aphasic patients, we develop PRISM (Perturbation-based Regional Interpretability through Subtraction Mapping). PRISM maps the seven clinical Philadelphia Naming Test categories, subtracts error classes pairwise, and treats each perturbation seed as a subject in a group analysis with threshold-free cluster enhancement along the layer axis. We run a structurally matched analysis on 213 chronic post-stroke aphasia patients using correlation-difference lesion-symptom mapping, and replicate both sides on held-out splits. The designs match in subject dimension (seeds, patients), spatial dimension (layers, atlas-parcellated cortex) and thresholding, but the contrast operator differs: a within-subject error-proportion difference for the LLM, a between-subject correlation difference for the cortex. Both substrates recover a robust phonemic-favoring dissociation, a deep layer cluster and a frontal-perisylvian cortical cluster, both replicating; the semantic-favoring direction is a consistently signed but non-significant trend on both. PRISM thus gives a falsifiable, spatially resolved test of functional-specialization claims in transformer language models. A confirmatory ROI-level intervention (PRISM Stage 3) licensing the strongest causal-mechanism claim is left to subsequent work.

Related papers

• CofactVLA: Deconfounding Vision-Language-Action Models via Counterfactual Intervention [48.20065816234311]
  Vision-Language-Action (VLA) models have driven significant progress in robotic manipulation, yet they struggle with the vision-override phenomenon.<n>We propose CofactVLA, a novel causal intervention framework.<n>We show that CofactVLA establishes a new state-of-the-art across diverse simulation benchmarks.
  arXiv  Detail & Related papers  (2026-08-05T02:58:41Z)
• Look-Closer-Then-Diagnose: Confidence-Aware Ultrasound VQA via Active Zooming [41.40761036883243]
  Vision-Language Models (VLMs) have significantly advanced medical visual question answering, yet their performance in ultrasound remains suboptimal.<n>We propose a framework specifically designed to consider the sonographer's cognitive workflow.<n>We first introduce a structured Zoom-then-Diagnose paradigm, which replicates the interactive search process to enable lesion-focused reasoning.
  arXiv  Detail & Related papers  (2026-05-20T19:06:34Z)
• Stroke Lesions as a Rosetta Stone for Language Model Interpretability [6.528508321422611]
  We present the Brain-LLM Unified Model (BLUM) as an external reference structure for evaluating large language models.<n>Using data from individuals with chronic post-stroke aphasia, we trained symptom-to-lesion models that predict brain damage location from behavioral error profiles.<n>BLUM error profiles were sufficiently similar to human error profiles that predicted lesions corresponded to actual lesions in error-matched humans above chance.
  arXiv  Detail & Related papers  (2026-02-03T23:22:37Z)
• Component-Level Lesioning of Language Models Reveals Clinically Aligned Aphasia Phenotypes [40.41503864764337]
  We introduce a component-level framework that simulates aphasia by selectively perturbing functional components in large language models.<n>Our pipeline identifies subtype-linked components for Broca's and Wernicke's aphasia, and induces graded impairments by progressively perturbing the top-k subtype-linked components.<n>Across architectures and lesioning strategies, subtype-targeted perturbations yield more systematic, aphasia-like regressions than size-matched random perturbations.
  arXiv  Detail & Related papers  (2026-01-27T15:47:22Z)
• PTSM: Physiology-aware and Task-invariant Spatio-temporal Modeling for Cross-Subject EEG Decoding [29.645617313721186]
  Cross-subject electroencephalography (EEG) decoding remains a fundamental challenge in brain-computer interface (BCI) research.<n>This paper proposed PTSM, a novel framework for interpretable and robust EEG decoding across unseen subjects.<n> PTSM employs a dual-branch mechanism that independently learns personalized and shared-temporal patterns.
  arXiv  Detail & Related papers  (2025-08-15T09:51:14Z)
• Signal in the Noise: Polysemantic Interference Transfers and Predicts Cross-Model Influence [46.548276232795466]
  Polysemanticity is pervasive in language models and remains a major challenge for interpretation and model behavioral control.<n>We map the polysemantic topology of two small models to identify feature pairs that are semantically unrelated yet exhibit interference within models.<n>We intervene at four loci (prompt, token, feature, neuron) and measure induced shifts in the next-token prediction distribution, uncovering polysemantic structures that expose a systematic vulnerability in these models.
  arXiv  Detail & Related papers  (2025-05-16T18:20:42Z)
• Interpretable Spatio-Temporal Embedding for Brain Structural-Effective Network with Ordinary Differential Equation [56.34634121544929]
  In this study, we first construct the brain-effective network via the dynamic causal model. We then introduce an interpretable graph learning framework termed Spatio-Temporal Embedding ODE (STE-ODE) This framework incorporates specifically designed directed node embedding layers, aiming at capturing the dynamic interplay between structural and effective networks.
  arXiv  Detail & Related papers  (2024-05-21T20:37:07Z)
• Deep Variational Lesion-Deficit Mapping [0.3914676152740142]
  We introduce a comprehensive framework for lesion-deficit model comparison. We show that our model outperforms established methods by a substantial margin across all simulation scenarios. Our analysis justifies the widespread adoption of this approach.
  arXiv  Detail & Related papers  (2023-05-27T13:49:35Z)
• A comprehensive comparative evaluation and analysis of Distributional Semantic Models [61.41800660636555]
  We perform a comprehensive evaluation of type distributional vectors, either produced by static DSMs or obtained by averaging the contextualized vectors generated by BERT. The results show that the alleged superiority of predict based models is more apparent than real, and surely not ubiquitous. We borrow from cognitive neuroscience the methodology of Representational Similarity Analysis (RSA) to inspect the semantic spaces generated by distributional models.
  arXiv  Detail & Related papers  (2021-05-20T15:18:06Z)
• Going Beyond Saliency Maps: Training Deep Models to Interpret Deep Models [16.218680291606628]
  Interpretability is a critical factor in applying complex deep learning models to advance the understanding of brain disorders. We propose to train simulator networks that can warp a given image to inject or remove patterns of the disease. We apply our approach to interpreting classifiers trained on a synthetic dataset and two neuroimaging datasets to visualize the effect of the Alzheimer's disease and alcohol use disorder.
  arXiv  Detail & Related papers  (2021-02-16T15:57:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.