Linguistic properties and model scale in brain encoding: from small to compressed language models

• URL: http://arxiv.org/abs/2602.07547v1
• Date: Sat, 07 Feb 2026 13:48:45 GMT
• Title: Linguistic properties and model scale in brain encoding: from small to compressed language models
• Authors: Subba Reddy Oota, Vijay Rowtula, Satya Sai Srinath Namburi, Khushbu Pahwa, Anant Khandelwal, Manish Gupta, Tanmoy Chakraborty, Bapi S. Raju,
• Abstract summary: Recent work has shown that scaling large language models (LLMs) improves their alignment with human brain activity.<n>This raises a fundamental question: what is the minimal model capacity required to capture brain-relevant representations?<n>We investigate how constraining model scale and numerical precision affects brain alignment.
• Score: 39.040386424307165
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent work has shown that scaling large language models (LLMs) improves their alignment with human brain activity, yet it remains unclear what drives these gains and which representational properties are responsible. Although larger models often yield better task performance and brain alignment, they are increasingly difficult to analyze mechanistically. This raises a fundamental question: what is the minimal model capacity required to capture brain-relevant representations? To address this question, we systematically investigate how constraining model scale and numerical precision affects brain alignment. We compare full-precision LLMs, small language models (SLMs), and compressed variants (quantized and pruned) by predicting fMRI responses during naturalistic language comprehension. Across model families up to 14B parameters, we find that 3B SLMs achieve brain predictivity indistinguishable from larger LLMs, whereas 1B models degrade substantially, particularly in semantic language regions. Brain alignment is remarkably robust to compression: most quantization and pruning methods preserve neural predictivity, with GPTQ as a consistent exception. Linguistic probing reveals a dissociation between task performance and brain predictivity: compression degrades discourse, syntax, and morphology, yet brain predictivity remains largely unchanged. Overall, brain alignment saturates at modest model scales and is resilient to compression, challenging common assumptions about neural scaling and motivating compact models for brain-aligned language modeling.

Related papers

• Abstraction Induces the Brain Alignment of Language and Speech Models [4.102097905864135]
  We show that the correspondence between speech and language models and the brain derives from shared meaning abstraction.<n>We show that a layer's intrinsic dimension strongly predicts how well it explains fMRI and ECoG signals.
  arXiv  Detail & Related papers  (2026-02-03T23:35:29Z)
• Do Large Language Models Think Like the Brain? Sentence-Level Evidence from fMRI and Hierarchical Embeddings [28.210559128941593]
  This study investigates how hierarchical representations in large language models align with the dynamic neural responses during human sentence comprehension.<n>Results show that improvements in model performance drive the evolution of representational architectures toward brain-like hierarchies.
  arXiv  Detail & Related papers  (2025-05-28T16:40:06Z)
• Generative causal testing to bridge data-driven models and scientific theories in language neuroscience [82.995061475971]
  We present generative causal testing (GCT), a framework for generating concise explanations of language selectivity in the brain.<n>We show that GCT can dissect fine-grained differences between brain areas with similar functional selectivity.
  arXiv  Detail & Related papers  (2024-10-01T15:57:48Z)
• Brain-Like Language Processing via a Shallow Untrained Multihead Attention Network [16.317199232071232]
  Large Language Models (LLMs) have been shown to be effective models of the human language system. In this work, we investigate the key architectural components driving the surprising alignment of untrained models.
  arXiv  Detail & Related papers  (2024-06-21T12:54:03Z)
• fMRI predictors based on language models of increasing complexity recover brain left lateralization [4.1618731507412505]
  We show that the left-right difference in brain correlation follows a scaling law with the number of parameters. This finding reconciles computational analyses of brain activity using large language models with the classic observation from aphasic patients showing left hemisphere dominance for language.
  arXiv  Detail & Related papers  (2024-05-28T09:24:52Z)
• MindBridge: A Cross-Subject Brain Decoding Framework [60.58552697067837]
  Brain decoding aims to reconstruct stimuli from acquired brain signals. Currently, brain decoding is confined to a per-subject-per-model paradigm. We present MindBridge, that achieves cross-subject brain decoding by employing only one model.
  arXiv  Detail & Related papers  (2024-04-11T15:46:42Z)
• In-Context Language Learning: Architectures and Algorithms [73.93205821154605]
  We study ICL through the lens of a new family of model problems we term in context language learning (ICLL) We evaluate a diverse set of neural sequence models on regular ICLL tasks.
  arXiv  Detail & Related papers  (2024-01-23T18:59:21Z)
• Scaling laws for language encoding models in fMRI [47.498241053872924]
  We tested whether larger open-source models are better at predicting brain responses recorded using fMRI. Similar logarithmic behavior was observed when scaling the size of the fMRI training set. These results suggest that increasing scale in both models and data will yield incredibly effective models of language processing in the brain.
  arXiv  Detail & Related papers  (2023-05-19T17:53:03Z)
• Neural Language Models are not Born Equal to Fit Brain Data, but Training Helps [75.84770193489639]
  We examine the impact of test loss, training corpus and model architecture on the prediction of functional Magnetic Resonance Imaging timecourses of participants listening to an audiobook. We find that untrained versions of each model already explain significant amount of signal in the brain by capturing similarity in brain responses across identical words. We suggest good practices for future studies aiming at explaining the human language system using neural language models.
  arXiv  Detail & Related papers  (2022-07-07T15:37:17Z)
• Model-based analysis of brain activity reveals the hierarchy of language in 305 subjects [82.81964713263483]
  A popular approach to decompose the neural bases of language consists in correlating, across individuals, the brain responses to different stimuli. Here, we show that a model-based approach can reach equivalent results within subjects exposed to natural stimuli.
  arXiv  Detail & Related papers  (2021-10-12T15:30:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.