The Giant Hippocampus: From Structural Monoculture to a System of Systems

• URL: http://arxiv.org/abs/2607.19973v1
• Date: Wed, 22 Jul 2026 09:55:25 GMT
• Title: The Giant Hippocampus: From Structural Monoculture to a System of Systems
• Abstract summary: AI researchers describe state-of-the-art models as one thing repeated at scale: the Transformer, wired identically for text, pixels, or speech.<n>Neuroscientists describe the cortex as a mosaic - dense Layer 4 in visual cortex for spatial encoding, thick Layers 5/6 in motion cortex for temporal integration.<n>This paper argues the gap is a structural error, not a stylistic one, and is measurable.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: AI researchers describe state-of-the-art models as one thing repeated at scale: the Transformer, wired identically for text, pixels, or speech. Neuroscientists describe the cortex as a mosaic - dense Layer 4 in visual cortex for spatial encoding, thick Layers 5/6 in motion cortex for temporal integration - different jobs solved by different structures. This paper argues the gap is a structural error, not a stylistic one, and is measurable. A century of cytoarchitecture, from Brodmann to single-cell Patch-seq, shows distinct cognitive functions are implemented by qualitatively different structures, not by rescaling one template. The convolutional neural network is the field's own proof: local receptive fields and hierarchical depth encoded this prior directly, reaching strong image recognition on far less data than later architectures needed. The paper traces how this lesson was discarded: the "Hardware Lottery" made the Transformer the path of least resistance, not the principled choice, and Mixture-of-Experts, often cited as diversity, in fact partitions parameters among identical experts. A functionalist analysis shows the Transformer is best understood as a functional analog of the hippocampal formation, not a general-purpose cortex - the same mistake as treating cortex as one giant Broca's area, except the field has now standardized on a giant hippocampus, applied to tasks it was never built for: audition, executive gating, working memory. The paper closes with an alternative: a Heterogeneous Topological Network, a System of Systems in which distinct modules keep the inductive bias their computation demands and communicate through standardized interfaces. This is a design discipline for AI architects, not cognitive science: specify modularity before training, using structural evidence as a design input rather than reverse-engineering architecture from a trained model's behavior.

Related papers

• Another Blueprint In The Wall: How to Ask Frontier AI Like a Kid? [0.16921396880325779]
  This paper reports experiments across six frontier model types from OpenAI, Anthropic, xAI, and Google DeepMind.<n>Ten independent sessions per model type used the same three stage prompt sequence, progressing from architectural preference to a full ASCII backbone.<n>Most runs remained close to this common structure, while a small number developed markedly greater engineering specificity.
  arXiv  Detail & Related papers  (2026-09-13T21:32:35Z)
• An expressivity analysis of hierarchical modelling in deep transformers via bounded-depth grammars [54.11540943172608]
  Deep neural networks are widely believed to derive their expressive power from their ability to form textbfhierarchical representations<n>In language modeling, textbftransformers have emerged as the dominant architecture, with early layers capturing local syntactic patterns and later layers encoding more complex clause-level dependencies.<n>We argue that these architectures possess the structural capacity to encode abstract grammatical states into low-dimensional, linearly separable subspaces within the residual stream.
  arXiv  Detail & Related papers  (2026-06-16T05:02:13Z)
• The Missing Knowledge Layer in Cognitive Architectures for AI Agents [0.0]
  Two most influential cognitive architecture frameworks for AI agents, CoALA and JEPA, both lack an explicit Knowledge layer with its own persistence semantics.<n>We propose a four-layer decom position (Knowledge, Memory, Wisdom, Intelligence) where each layer has fundamentally different persistence semantics.<n>We argue that these distinctions demand distinct persistence semantics in engineering implementations, and that no current framework or system provides this.
  arXiv  Detail & Related papers  (2026-04-13T12:05:30Z)
• DDCL-INCRT: A Self-Organising Transformer with Hierarchical Prototype Structure (Theoretical Foundations) [0.0]
  This paper introduces DDCL-INCRT, an architecture that determines its own structure during training.<n>Main theoretical finding is that these two mechanisms reinforce each other.<n>At convergence, the network self-organises into a hierarchy of heads ordered by representational granularity.
  arXiv  Detail & Related papers  (2026-04-02T10:39:06Z)
• Bearing Syntactic Fruit with Stack-Augmented Neural Networks [59.49467149799849]
  We show that stack-augmented neural networks are more accurate models of human language acquisition than standard architectures.<n>We also propose a modification to the stack RNN architecture that improves hierarchical generalization.
  arXiv  Detail & Related papers  (2025-11-05T15:30:58Z)
• Grokking of Hierarchical Structure in Vanilla Transformers [72.45375959893218]
  We show that transformer language models can learn to generalize hierarchically after training for extremely long periods. intermediate-depth models generalize better than both very deep and very shallow transformers.
  arXiv  Detail & Related papers  (2023-05-30T04:34:13Z)
• How Do Transformers Learn Topic Structure: Towards a Mechanistic Understanding [56.222097640468306]
  We provide mechanistic understanding of how transformers learn "semantic structure" We show, through a combination of mathematical analysis and experiments on Wikipedia data, that the embedding layer and the self-attention layer encode the topical structure.
  arXiv  Detail & Related papers  (2023-03-07T21:42:17Z)
• System identification of neural systems: If we got it right, would we know? [4.414362557866993]
  We evaluate the most commonly used comparison techniques to identify a model by replacing brain recordings with known ground truth models. We show the limitations of using functional similarity scores in identifying higher-level architectural motifs.
  arXiv  Detail & Related papers  (2023-02-13T20:32:37Z)
• Equivariance with Learned Canonicalization Functions [77.32483958400282]
  We show that learning a small neural network to perform canonicalization is better than using predefineds. Our experiments show that learning the canonicalization function is competitive with existing techniques for learning equivariant functions across many tasks.
  arXiv  Detail & Related papers  (2022-11-11T21:58:15Z)
• Dynamic Inference with Neural Interpreters [72.90231306252007]
  We present Neural Interpreters, an architecture that factorizes inference in a self-attention network as a system of modules. inputs to the model are routed through a sequence of functions in a way that is end-to-end learned. We show that Neural Interpreters perform on par with the vision transformer using fewer parameters, while being transferrable to a new task in a sample efficient manner.
  arXiv  Detail & Related papers  (2021-10-12T23:22:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.