Heterogeneous Neural Predictivity from Language Models During Naturalistic Comprehension

• URL: http://arxiv.org/abs/2606.26880v1
• Date: Thu, 25 Jun 2026 11:08:49 GMT
• Title: Heterogeneous Neural Predictivity from Language Models During Naturalistic Comprehension
• Abstract summary: Language-model representations provide structured, high-dimensional annotations of naturalistic language stimuli.<n>Positive held-out prediction and gains over low-level baselines were widespread in source-level summaries.<n>These findings show that language-model-derived quantities can annotate neural activity during natural speech and text comprehension.
• Score: 0.6331422666055228
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Language-model representations provide structured, high-dimensional annotations of naturalistic language stimuli and can serve as informative neural predictors during comprehension. We analyzed locked derived data from Brain Treebank, MEG-MASC, and Podcast ECoG with eight frozen language models, blocked encoding models, and matched temporal, nuisance, and representation-capacity controls. Positive held-out prediction and gains over low-level baselines were widespread in source-level summaries. Across Brain Treebank and Podcast ECoG, 67 of 432 evaluable rows met a controlled predictive-only criterion, and model-side feature ablations changed prediction scores in most evaluable source rows. Brain-derived, timing-linked, acoustic, and implanted-signal controls confirmed component-level sensitivity of the analysis pipeline. These findings show that language-model-derived quantities can annotate neural activity during natural speech and text comprehension. Participant-level matched-control advantages were localized rather than uniform, response-profile and feature-specificity contrasts bounded representational or computational interpretations, and complete co-indexed integrated interpretation will require future jointly indexed coverage. Together, the analyses identify language-model features as useful neural predictors and separate predictive usefulness from claims about shared neural organization or language-processing computations.

Related papers

• RABBiT: Rapidly adaptive BOLD foundation model via brain-tuning for accurate zero-shot and few-shot prediction of speech-elicited responses in the brain [9.911908326274581]
  We introduce RABBiT, a compact audio-to-fMRI encoder for accurate zero- and few-shot prediction.<n>RABBiT enables scalable population-level analyses of language in the human brain.
  arXiv  Detail & Related papers  (2026-07-06T14:54:19Z)
• Cross-Subject Semantic Decoding with Shared-Space Alignment for Generalized Neural Representation Learning [80.18561652862725]
  Generalizing across subjects remains challenging in invasive neural recordings.<n>We propose a cross-subject semantic decoding framework that aligns neural responses to speech perception from multiple subjects into a shared latent space.
  arXiv  Detail & Related papers  (2026-07-03T03:59:16Z)
• Interpreting Brain Responses to Language with Sparse Features from Language Models [5.261667457257713]
  We develop an encoding framework that replaces dense LM hidden states with hierarchically-organized autoencoder features.<n>We show that brain responses during language processing are not merely predictable from an arbitrary set of LM features.
  arXiv  Detail & Related papers  (2026-06-05T03:01:28Z)
• Mechanistic Understanding and Mitigation of Language Confusion in English-Centric Large Language Models [56.61984030508691]
  We present the first mechanistic interpretability study of language confusion.<n>We show that confusion points (CPs) are central to this phenomenon.<n>We show that editing a small set of critical neurons, identified via comparative analysis with a multilingual-tuned counterpart, substantially mitigates confusion.
  arXiv  Detail & Related papers  (2025-05-22T11:29:17Z)
• Interpretable Next-token Prediction via the Generalized Induction Head [59.500195503897764]
  Generalized Induction-Head Model (GIM) is an interpretable model for next-token prediction.<n>In language modeling, GIM improves next-token prediction by up to 25%p over interpretable baselines.<n>In an fMRI setting, GIM improves neural response prediction by 20%.
  arXiv  Detail & Related papers  (2024-10-31T12:33:26Z)
• A predictive learning model can simulate temporal dynamics and context effects found in neural representations of continuous speech [11.707968216076075]
  Recent work in cognitive neuroscience has identified temporal and contextual characteristics in humans' neural encoding of speech. In this study, we simulated similar analyses with representations extracted from a computational model that was trained on unlabelled speech. Our simulations revealed temporal dynamics similar to those in brain signals, implying that these properties can arise without linguistic knowledge.
  arXiv  Detail & Related papers  (2024-05-13T23:36:19Z)
• BrainLLM: Generative Language Decoding from Brain Recordings [77.66707255697706]
  We propose a generative language BCI that utilizes the capacity of a large language model and a semantic brain decoder.<n>The proposed model can generate coherent language sequences aligned with the semantic content of visual or auditory language stimuli.<n>Our findings demonstrate the potential and feasibility of employing BCIs in direct language generation.
  arXiv  Detail & Related papers  (2023-11-16T13:37:21Z)
• Modeling structure-building in the brain with CCG parsing and large language models [9.17816011606258]
  Combinatory Categorial Grammars (CCGs) are sufficiently expressive directly compositional models of grammar. We evaluate whether a more expressive CCG provides a better model than a context-free grammar for human neural signals collected with fMRI.
  arXiv  Detail & Related papers  (2022-10-28T14:21:29Z)
• Dependency-based Mixture Language Models [53.152011258252315]
  We introduce the Dependency-based Mixture Language Models. In detail, we first train neural language models with a novel dependency modeling objective. We then formulate the next-token probability by mixing the previous dependency modeling probability distributions with self-attention.
  arXiv  Detail & Related papers  (2022-03-19T06:28:30Z)
• Demystifying Neural Language Models' Insensitivity to Word-Order [7.72780997900827]
  We investigate the insensitivity of natural language models to word-order by quantifying perturbations. We find that neural language models require local ordering more so than the global ordering of tokens.
  arXiv  Detail & Related papers  (2021-07-29T13:34:20Z)
• Mechanisms for Handling Nested Dependencies in Neural-Network Language Models and Humans [75.15855405318855]
  We studied whether a modern artificial neural network trained with "deep learning" methods mimics a central aspect of human sentence processing. Although the network was solely trained to predict the next word in a large corpus, analysis showed the emergence of specialized units that successfully handled local and long-distance syntactic agreement. We tested the model's predictions in a behavioral experiment where humans detected violations in number agreement in sentences with systematic variations in the singular/plural status of multiple nouns.
  arXiv  Detail & Related papers  (2020-06-19T12:00:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.