An Existence Proof for Neural Language Models That Can Explain Garden-Path Effects via Surprisal

• URL: http://arxiv.org/abs/2604.18293v1
• Date: Mon, 20 Apr 2026 14:01:11 GMT
• Title: An Existence Proof for Neural Language Models That Can Explain Garden-Path Effects via Surprisal
• Abstract summary: We investigate whether it is impossible to construct a neural LM that can explain garden-path effects via surprisal.<n>Our results show that fine-tuned LMs do not overfit and successfully capture human reading slowdowns on held-out garden-path items.
• Score: 21.29543663048103
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Surprisal theory hypothesizes that the difficulty of human sentence processing increases linearly with surprisal, the negative log-probability of a word given its context. Computational psycholinguistics has tested this hypothesis using language models (LMs) as proxies for human prediction. While surprisal derived from recent neural LMs generally captures human processing difficulty on naturalistic corpora that predominantly consist of simple sentences, it severely underestimates processing difficulty on sentences that require syntactic disambiguation (garden-path effects). This leads to the claim that the processing difficulty of such sentences cannot be reduced to surprisal, although it remains possible that neural LMs simply differ from humans in next-word prediction. In this paper, we investigate whether it is truly impossible to construct a neural LM that can explain garden-path effects via surprisal. Specifically, instead of evaluating off-the-shelf neural LMs, we fine-tune these LMs on garden-path sentences so as to better align surprisal-based reading-time estimates with actual human reading times. Our results show that fine-tuned LMs do not overfit and successfully capture human reading slowdowns on held-out garden-path items; they even improve predictive power for human reading times on naturalistic corpora and preserve their general LM capabilities. These results provide an existence proof for a neural LM that can explain both garden-path effects and naturalistic reading times via surprisal, but also raise a theoretical question: what kind of evidence can truly falsify surprisal theory?

Related papers

• Why are language models less surprised than humans? Testing the Parse Multiplicity Mismatch Hypothesis [14.478855009044807]
  Surprisal theory posits that the processing difficulty of a word is determined by its predictability in context.<n> LMs successfully predict reading times in naturalistic text, but systematically underpredict the magnitude of difficulty observed in controlled studies of syntactic ambiguity.<n>This suggests that differences in the number of simultaneous parses available to LMs and humans cannot reconcile LM-based surprisal with human sentence processing.
  arXiv  Detail & Related papers  (2026-05-14T21:44:26Z)
• Dual Alignment Between Language Model Layers and Human Sentence Processing [21.60204953623752]
  We explore internal layers that better estimate human cognitive effort observed in syntactic ambiguity processing in English.<n>Experiments show that, in contrast to naturalistic reading, later layers better estimate such a cognitive effort, but still underestimate the human data.
  arXiv  Detail & Related papers  (2026-04-20T17:51:26Z)
• Language Statistics and False Belief Reasoning: Evidence from 41 Open-Weight LMs [6.600578957536851]
  We assess LM mental state reasoning behavior across 41 open-weight models.<n>We find sensitivity to implied knowledge states in 34% of the LMs tested.<n>Larger LMs show increased sensitivity and also exhibit higher psychometric predictive power.
  arXiv  Detail & Related papers  (2026-02-17T23:20:08Z)
• H-Neurons: On the Existence, Impact, and Origin of Hallucination-Associated Neurons in LLMs [56.31565301428888]
  We identify hallucination-associated neurons (H-Neurons) in large language models (LLMs)<n>In terms of identification, we demonstrate that a remarkably sparse subset of neurons can reliably predict hallucination occurrences.<n>In terms of behavioral impact, controlled interventions reveal that these neurons are causally linked to over-compliance behaviors.
  arXiv  Detail & Related papers  (2025-12-01T15:32:14Z)
• Hallucinations are inevitable but can be made statistically negligible. The "innate" inevitability of hallucinations cannot explain practical LLM issues [23.12905996928255]
  Hallucinations, a phenomenon where a language model (LM) generates nonfactual content, pose a significant challenge to the practical deployment of LMs.<n>Recent studies established a computability-theoretic result showing that any LM will inevitably generate hallucinations on an infinite set of inputs.<n>This paper claims that those "innate" inevitability results from computability theory and diagonal argument, in principle, cannot explain practical issues of LMs.
  arXiv  Detail & Related papers  (2025-02-15T07:28:40Z)
• Alleviating Hallucinations of Large Language Models through Induced Hallucinations [67.35512483340837]
  Large language models (LLMs) have been observed to generate responses that include inaccurate or fabricated information. We propose a simple textitInduce-then-Contrast Decoding (ICD) strategy to alleviate hallucinations.
  arXiv  Detail & Related papers  (2023-12-25T12:32:49Z)
• Divergences between Language Models and Human Brains [59.100552839650774]
  We systematically explore the divergences between human and machine language processing.<n>We identify two domains that LMs do not capture well: social/emotional intelligence and physical commonsense.<n>Our results show that fine-tuning LMs on these domains can improve their alignment with human brain responses.
  arXiv  Detail & Related papers  (2023-11-15T19:02:40Z)
• Reducing Hallucinations in Neural Machine Translation with Feature Attribution [54.46113444757899]
  We present a case study focusing on model understanding and regularisation to reduce hallucinations in NMT. We first use feature attribution methods to study the behaviour of an NMT model that produces hallucinations. We then leverage these methods to propose a novel loss function that substantially helps reduce hallucinations and does not require retraining the model from scratch.
  arXiv  Detail & Related papers  (2022-11-17T20:33:56Z)
• Syntactic Surprisal From Neural Models Predicts, But Underestimates, Human Processing Difficulty From Syntactic Ambiguities [19.659811811023374]
  We propose a method for estimating syntactic predictability from a language model. We find that treating syntactic predictability independently from lexical predictability indeed results in larger estimates of garden path effects. Our results support the hypothesis that predictability is not the only factor responsible for the processing cost associated with garden path sentences.
  arXiv  Detail & Related papers  (2022-10-21T18:30:56Z)
• Neural Language Models are not Born Equal to Fit Brain Data, but Training Helps [75.84770193489639]
  We examine the impact of test loss, training corpus and model architecture on the prediction of functional Magnetic Resonance Imaging timecourses of participants listening to an audiobook. We find that untrained versions of each model already explain significant amount of signal in the brain by capturing similarity in brain responses across identical words. We suggest good practices for future studies aiming at explaining the human language system using neural language models.
  arXiv  Detail & Related papers  (2022-07-07T15:37:17Z)
• Long-range and hierarchical language predictions in brains and algorithms [82.81964713263483]
  We show that while deep language algorithms are optimized to predict adjacent words, the human brain would be tuned to make long-range and hierarchical predictions. This study strengthens predictive coding theory and suggests a critical role of long-range and hierarchical predictions in natural language processing.
  arXiv  Detail & Related papers  (2021-11-28T20:26:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.