When Is a Steerable Concept Representation Real? Measurement Confounds in a Cross-Family Audit of Neuroscience Parallels in LLMs

• URL: http://arxiv.org/abs/2608.08159v1
• Date: Sat, 08 Aug 2026 14:37:53 GMT
• Title: When Is a Steerable Concept Representation Real? Measurement Confounds in a Cross-Family Audit of Neuroscience Parallels in LLMs
• Abstract summary: Large language models (LLMs) are increasingly reported to exhibit human-like neural and cognitive signatures.<n>Our main experiment examines the causal steerability of concept directions.<n>With raw activation units and a fixed layer and coefficient, steerability appears to increase with model scale.<n>With residual-norm-comparable interventions and held-out operating-point selection, concept steering remains significant at every scale.
• Score: 8.657685524975246
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) are increasingly reported to exhibit human-like neural and cognitive signatures, including concept cells, mental number lines, and cognitive maps. These claims often rely on linear probing and activation steering applied to a single model, yet both methods are highly sensitive to measurement choices. A reported parallel may therefore reflect the model, the measurement procedure, or both. We audit four representative neuroscience-inspired paradigms across 17 models from five families, spanning $0.6$B to $72$B parameters. Our main experiment examines the causal steerability of concept directions. With raw activation units and a fixed layer and coefficient, steerability appears to increase with model scale, resembling an emergent capability. However, this pattern is produced by an uncalibrated pipeline rather than by a claim established in the steering literature. The trend depends jointly on raw units, the readout metric, and the operating point; correcting any one of these removes it. With residual-norm-comparable interventions and held-out operating-point selection, concept steering remains significant at every scale, but shows no significant trend across the Qwen3 series, although the confidence interval does not rule out a moderate positive slope. The remaining results are mixed. A linear geographic world map is consistently decodable in every tested checkpoint up to $72$B. Number magnitude is strongly encoded, but whether individual neurons appear bell-shaped or monotonic depends on the selection criterion. Language-specific structure is localizable, but the direction of the cross-lingual asymmetry reverses under a different attribution method. These results suggest that the main constraint on AI neuroscience is not a lack of phenomena, but a lack of comparable measurements and adequate controls. We release the protocol, stimuli, and code.

Related papers

• Intrinsic Structure: Spectral Identifiability for Mechanistic Interpretability [51.56484100374058]
  We prove the first identifiability theorem for a mechanistic-interpretability primitive.<n>The spectrum is an identifiable, model-intrinsic fingerprint with a stated error bar, not a legible decomposition.
  arXiv  Detail & Related papers  (2026-08-10T19:42:01Z)
• Representational Depth of Evaluation Awareness Shifts With Scale in Open-Weight Language Models [0.0]
  A model that recognises an evaluation context could alter its behaviour strategically, making downstream benchmarks harder to interpret.<n>Using 11 models spanning Qwen 2.5, Gemma 2, and Llama 3.2, we find a systematic size-dependent shift in representational depth.<n>This suggests that scale changes not only the strength of evaluation-awareness but also where it is most linearly recoverable in the network.
  arXiv  Detail & Related papers  (2026-06-28T04:48:17Z)
• Leverage Is Not Reach: A Control-Window Law for Single-Neuron Steering in Language Models [1.7259824817932294]
  We develop a budget normalized control window framework for single neuron steering.<n>A dose along one write direction reduces to one control coordinate.<n>Coherent control exists when a behavior trigger lies below the collapse ceiling.
  arXiv  Detail & Related papers  (2026-06-18T06:25:18Z)
• Before the First Token: Scale-Dependent Emergence of Hallucination Signals in Autoregressive Language Models [6.5607014897534865]
  We study the temporal dynamics of hallucination-indicative internal representations across 7 autoregressive transformers.<n>We identify a scale-dependent phase transition: models below 400M parameters show chance-level probe accuracy at every generation position.<n>At the 7B scale, we observe a striking dissociation: Pythia-6.9B produces a flat temporal profile, while instruction-tuned Qwen2.5-7B shows a dominant pre-generation effect.
  arXiv  Detail & Related papers  (2026-03-20T02:30:01Z)
• Anatomy of a Lie: A Multi-Stage Diagnostic Framework for Tracing Hallucinations in Vision-Language Models [62.932580559941414]
  Vision-Language Models (VLMs) frequently "hallucinate" - generate plausible yet factually incorrect statements.<n>We propose a new paradigm for diagnosing hallucinations, recasting them from static output errors into dynamic pathologies of a model's computational cognition.
  arXiv  Detail & Related papers  (2026-03-16T17:20:38Z)
• Language Model Circuits Are Sparse in the Neuron Basis [50.460651620833055]
  We show that textbfMLP neurons are as sparse a feature basis as SAEs.<n>This work advances automated interpretability of language models without additional training costs.
  arXiv  Detail & Related papers  (2026-01-30T05:41:19Z)
• Superposition disentanglement of neural representations reveals hidden alignment [6.015414975356222]
  In neuroscience and AI, representational alignment metrics measure the extent to which different deep neural networks (DNNs) or brains represent similar information.<n>We develop a theory for how the strict permutation metrics are dependent on superposition arrangements.<n>Our results suggest that superposition disentanglement is necessary for mapping metrics to uncover the true representational alignment between neural codes.
  arXiv  Detail & Related papers  (2025-10-03T17:12:40Z)
• An Orthogonal Learner for Individualized Outcomes in Markov Decision Processes [55.93922317950527]
  We develop a novel meta-learner called DRQ-learner.<n>Our DRQ-learner is applicable to settings with both discrete and continuous state spaces.
  arXiv  Detail & Related papers  (2025-09-30T15:49:29Z)
• The Causal Neural Connection: Expressiveness, Learnability, and Inference [125.57815987218756]
  An object called structural causal model (SCM) represents a collection of mechanisms and sources of random variation of the system under investigation. In this paper, we show that the causal hierarchy theorem (Thm. 1, Bareinboim et al., 2020) still holds for neural models. We introduce a special type of SCM called a neural causal model (NCM), and formalize a new type of inductive bias to encode structural constraints necessary for performing causal inferences.
  arXiv  Detail & Related papers  (2021-07-02T01:55:18Z)
• Neural Unsupervised Semantic Role Labeling [48.69930912510414]
  We present the first neural unsupervised model for semantic role labeling. We decompose the task as two argument related subtasks, identification and clustering. Experiments on CoNLL-2009 English dataset demonstrate that our model outperforms previous state-of-the-art baseline.
  arXiv  Detail & Related papers  (2021-04-19T04:50:16Z)
• And/or trade-off in artificial neurons: impact on adversarial robustness [91.3755431537592]
  Presence of sufficient number of OR-like neurons in a network can lead to classification brittleness and increased vulnerability to adversarial attacks. We define AND-like neurons and propose measures to increase their proportion in the network. Experimental results on the MNIST dataset suggest that our approach holds promise as a direction for further exploration.
  arXiv  Detail & Related papers  (2021-02-15T08:19:05Z)
• Manifold GPLVMs for discovering non-Euclidean latent structure in neural data [5.949779668853555]
  A common problem in neuroscience is to elucidate the collective neural representations of behaviorally important variables. Here, we propose a new probabilistic latent variable model to simultaneously identify the latent state and the way each neuron contributes to its representation.
  arXiv  Detail & Related papers  (2020-06-12T19:08:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.