Dual Alignment Between Language Model Layers and Human Sentence Processing

• URL: http://arxiv.org/abs/2604.18563v1
• Date: Mon, 20 Apr 2026 17:51:26 GMT
• Title: Dual Alignment Between Language Model Layers and Human Sentence Processing
• Abstract summary: We explore internal layers that better estimate human cognitive effort observed in syntactic ambiguity processing in English.<n>Experiments show that, in contrast to naturalistic reading, later layers better estimate such a cognitive effort, but still underestimate the human data.
• Score: 21.60204953623752
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: A recent study (Kuribayashi et al., 2025) has shown that human sentence processing behavior, typically measured on syntactically unchallenging constructions, can be effectively modeled using surprisal from early layers of large language models (LLMs). This raises the question of whether such advantages of internal layers extend to more syntactically challenging constructions, where surprisal has been reported to underestimate human cognitive effort. In this paper, we begin by exploring internal layers that better estimate human cognitive effort observed in syntactic ambiguity processing in English. Our experiments show that, in contrast to naturalistic reading, later layers better estimate such a cognitive effort, but still underestimate the human data. This dual alignment sheds light on different modes of sentence processing in humans and LMs: naturalistic reading employs a somewhat weak prediction akin to earlier layers of LMs, while syntactically challenging processing requires more fully-contextualized representations, better modeled by later layers of LMs. Motivated by these findings, we also explore several probability-update measures using shallow and deep layers of LMs, showing a complementary advantage to single-layer's surprisal in reading time modeling.

Related papers

• Encoding EEG Signals to Examine Human-Like Next-Word Prediction Behaviour in Language Models [3.1487162256026573]
  Next-word predictability influences brain response, as recorded at millisecond resolution using electroencephalography (EEG)<n>We generate regressors for both humans and LMs based on two information measures, including top-1 prediction and surprisal, to predict event-related potential (ERP)<n>Our results indicate that only surprisal potentially correlates with language-processing ERPs, especially for open-class words with high semantic content.
  arXiv  Detail & Related papers  (2026-07-17T23:11:16Z)
• Probing for Reading Times [60.30314214800421]
  We probe language model representations for human reading times in English, Greek, Hebrew, Russian, and Turkish.<n>We find that the representations from early layers outperform surprisal in predicting early-pass measures such as first fixation and gaze duration.
  arXiv  Detail & Related papers  (2026-04-20T18:12:59Z)
• To Words and Beyond: Probing Large Language Models for Sentence-Level Psycholinguistic Norms of Memorability and Reading Times [4.5166266531313966]
  Large Language Models produce estimates of psycholinguistic norms that correlate with human judgments.<n>We extend this approach to the previously unstudied features of sentence memorability and reading times.
  arXiv  Detail & Related papers  (2026-03-12T16:10:27Z)
• Can Language Models Learn Typologically Implausible Languages? [62.823015163987996]
  Grammatical features across human languages show intriguing correlations often attributed to learning biases in humans.<n>We discuss how language models (LMs) allow us to better determine the role of domain-general learning biases in language universals.<n>We test LMs on an array of highly naturalistic but counterfactual versions of the English (head-initial) and Japanese (head-final) languages.
  arXiv  Detail & Related papers  (2025-02-17T20:40:01Z)
• Large Language Models Are Human-Like Internally [44.996518290660816]
  Recent cognitive modeling studies have reported that larger language models (LMs) exhibit a poorer fit to human reading behavior.<n>We argue that prior conclusions were skewed by an exclusive focus on the final layers of LMs.<n>Our analysis reveals that next-word probabilities derived from internal layers of larger LMs align with human sentence processing data as well as, or better than, those from smaller LMs.
  arXiv  Detail & Related papers  (2025-02-03T18:48:32Z)
• Evidence from fMRI Supports a Two-Phase Abstraction Process in Language Models [1.534667887016089]
  We show that intermediate hidden states extracted from large language models are able to predict measured brain response to natural language stimuli. We also demonstrate a strong correspondence between layerwise encoding performance and the intrinsic dimensionality of representations from LLMs.
  arXiv  Detail & Related papers  (2024-09-09T16:33:16Z)
• Entropy Guided Extrapolative Decoding to Improve Factuality in Large Language Models [55.45444773200529]
  Large language models (LLMs) exhibit impressive natural language capabilities but suffer from hallucination. Recent work has focused on decoding techniques to improve factuality during inference.
  arXiv  Detail & Related papers  (2024-04-14T19:45:35Z)
• Fantastic Semantics and Where to Find Them: Investigating Which Layers of Generative LLMs Reflect Lexical Semantics [50.982315553104975]
  We investigate the bottom-up evolution of lexical semantics for a popular large language model, namely Llama2. Our experiments show that the representations in lower layers encode lexical semantics, while the higher layers, with weaker semantic induction, are responsible for prediction. This is in contrast to models with discriminative objectives, such as mask language modeling, where the higher layers obtain better lexical semantics.
  arXiv  Detail & Related papers  (2024-03-03T13:14:47Z)
• Humans and language models diverge when predicting repeating text [52.03471802608112]
  We present a scenario in which the performance of humans and LMs diverges. Human and GPT-2 LM predictions are strongly aligned in the first presentation of a text span, but their performance quickly diverges when memory begins to play a role. We hope that this scenario will spur future work in bringing LMs closer to human behavior.
  arXiv  Detail & Related papers  (2023-10-10T08:24:28Z)
• Probing Large Language Models from A Human Behavioral Perspective [24.109080140701188]
  Large Language Models (LLMs) have emerged as dominant foundational models in modern NLP. The understanding of their prediction processes and internal mechanisms, such as feed-forward networks (FFN) and multi-head self-attention (MHSA) remains largely unexplored.
  arXiv  Detail & Related papers  (2023-10-08T16:16:21Z)
• Is Supervised Syntactic Parsing Beneficial for Language Understanding? An Empirical Investigation [71.70562795158625]
  Traditional NLP has long held (supervised) syntactic parsing necessary for successful higher-level semantic language understanding (LU) Recent advent of end-to-end neural models, self-supervised via language modeling (LM), and their success on a wide range of LU tasks, questions this belief. We empirically investigate the usefulness of supervised parsing for semantic LU in the context of LM-pretrained transformer networks.
  arXiv  Detail & Related papers  (2020-08-15T21:03:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.