Categorical Perception in Large Language Model Hidden States: Structural Warping at Digit-Count Boundaries

• URL: http://arxiv.org/abs/2603.28258v1
• Date: Mon, 30 Mar 2026 10:34:58 GMT
• Title: Categorical Perception in Large Language Model Hidden States: Structural Warping at Digit-Count Boundaries
• Abstract summary: Categorical perception (CP) -- enhanced discriminability at category boundaries -- is among the most studied phenomena in perceptual psychology.<n>This paper reports that analogous geometric warping occurs in the hidden-state representations of large language models (LLMs) processing Arabic numerals.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Categorical perception (CP) -- enhanced discriminability at category boundaries -- is among the most studied phenomena in perceptual psychology. This paper reports that analogous geometric warping occurs in the hidden-state representations of large language models (LLMs) processing Arabic numerals. Using representational similarity analysis across six models from five architecture families, the study finds that a CP-additive model (log-distance plus a boundary boost) fits the representational geometry better than a purely continuous model at 100% of primary layers in every model tested. The effect is specific to structurally defined boundaries (digit-count transitions at 10 and 100), absent at non-boundary control positions, and absent in the temperature domain where linguistic categories (hot/cold) lack a tokenisation discontinuity. Two qualitatively distinct signatures emerge: "classic CP" (Gemma, Qwen), where models both categorise explicitly and show geometric warping, and "structural CP" (Llama, Mistral, Phi), where geometry warps at the boundary but models cannot report the category distinction. This dissociation is stable across boundaries and is a property of the architecture, not the stimulus. Structural input-format discontinuities are sufficient to produce categorical perception geometry in LLMs, independently of explicit semantic category knowledge.

Related papers

• Does Syntax Matter? A Graph-Augmented Variational Topic Model for Computational Social Sciences [51.56484100374058]
  This paper introduces the Structural Contextual Probabilistic Topic Model ( SCPTM), an architecture that incorporates syntactic dependency relations into topic inference.<n>We evaluate seven topic modeling techniques across four corpora differing in register and discourse structure.<n>Results show that SCPTM's neural architecture yields substantial gains in document-topic alignment over generative baselines.
  arXiv  Detail & Related papers  (2026-09-07T17:39:06Z)
• Hierarchical Grading in Large Language Models [0.0]
  We introduce Graded Large Language Models (GLLMs)<n>GLLMs are a framework that equips the representation space of a transformer with a grading.<n>The construction extends the theory of graded neural networks and graded transformers to autoregressive language models.
  arXiv  Detail & Related papers  (2026-07-23T20:14:48Z)
• Bridging Structure and Appearance: Topological Features for Robust Self-Supervised Segmentation [8.584363058858935]
  Self-supervised semantic segmentation methods often fail when faced with appearance ambiguities.<n>We argue that this is due to an over-reliance on unstable, appearance-based features such as shadows, glare, and local textures.<n>We propose textbfGASeg, a novel framework that bridges appearance and geometry by leveraging stable topological information.
  arXiv  Detail & Related papers  (2025-12-30T05:34:28Z)
• Visual Language Hypothesis [14.062822951292402]
  We study visual representation learning from a structural and topological perspective.<n>We show that approximating the quotient also places structural demands on the model architecture.
  arXiv  Detail & Related papers  (2025-12-29T09:43:10Z)
• Optimizing against Infeasible Inclusions from Data for Semantic Segmentation through Morphology [58.17907376475596]
  State-of-the-art semantic segmentation models are typically optimized in a data-driven fashion.<n>InSeIn extracts explicit inclusion constraints that govern spatial class relations from the semantic segmentation training set at hand.<n>It then enforces a morphological yet differentiable loss that penalizes violations of these constraints during training to promote prediction feasibility.
  arXiv  Detail & Related papers  (2024-08-26T22:39:08Z)
• Zero-Shot 3D Shape Correspondence [67.18775201037732]
  We propose a novel zero-shot approach to computing correspondences between 3D shapes. We exploit the exceptional reasoning capabilities of recent foundation models in language and vision. Our approach produces highly plausible results in a zero-shot manner, especially between strongly non-isometric shapes.
  arXiv  Detail & Related papers  (2023-06-05T21:14:23Z)
• A Layered Architecture for Universal Causality [4.119151469153588]
  We propose a layered hierarchical architecture called UCLA (Universal Causality Layered Architecture) At the top-most level, causal interventions are modeledly using a simplicial category of ordinal numbers. At the second layer, causal models are defined by a graph-type category.
  arXiv  Detail & Related papers  (2022-12-18T00:53:19Z)
• Linear Connectivity Reveals Generalization Strategies [54.947772002394736]
  Some pairs of finetuned models have large barriers of increasing loss on the linear paths between them. We find distinct clusters of models which are linearly connected on the test loss surface, but are disconnected from models outside the cluster. Our work demonstrates how the geometry of the loss surface can guide models towards different functions.
  arXiv  Detail & Related papers  (2022-05-24T23:43:02Z)
• Bipartite polygon models: entanglement classes and their nonlocal behaviour [0.0]
  We investigate Hardy's nonlocal behavior within a broad class of operational theories, including the qubit state space.<n>Our results uncover an unexplored class of almost-quantum correlations that can be associated with an explicit operational model.
  arXiv  Detail & Related papers  (2022-05-11T11:34:10Z)
• Polynomial Networks in Deep Classifiers [55.90321402256631]
  We cast the study of deep neural networks under a unifying framework. Our framework provides insights on the inductive biases of each model. The efficacy of the proposed models is evaluated on standard image and audio classification benchmarks.
  arXiv  Detail & Related papers  (2021-04-16T06:41:20Z)
• Causal Intervention for Weakly-Supervised Semantic Segmentation [122.1846968696862]
  We aim to generate better pixel-level pseudo-masks by using only image-level labels. We propose a structural causal model to analyze the causalities among images, contexts, and class labels. Based on it, we develop a new method: Context Adjustment (CONTA), to remove the confounding bias in image-level classification.
  arXiv  Detail & Related papers  (2020-09-26T09:26:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.