Algorithmic Consequences of Particle Filters for Sentence Processing: Amplified Garden-Paths and Digging-In Effects

• URL: http://arxiv.org/abs/2603.11412v1
• Date: Thu, 12 Mar 2026 00:56:42 GMT
• Title: Algorithmic Consequences of Particle Filters for Sentence Processing: Amplified Garden-Paths and Digging-In Effects
• Abstract summary: Under surprisal theory, linguistic representations affect processing difficulty only through the bottleneck of surprisal.<n>Our best estimates of surprisal come from large language models, which have no explicit representation of structural ambiguity.<n>We show that particle filter models offer an alternative where structural hypotheses are explicitly represented as a finite set of particles.
• Score: 10.217186322563318
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Under surprisal theory, linguistic representations affect processing difficulty only through the bottleneck of surprisal. Our best estimates of surprisal come from large language models, which have no explicit representation of structural ambiguity. While LLM surprisal robustly predicts reading times across languages, it systematically underpredicts difficulty when structural expectations are violated -- suggesting that representations of ambiguity are causally implicated in sentence processing. Particle filter models offer an alternative where structural hypotheses are explicitly represented as a finite set of particles. We prove several algorithmic consequences of particle filter models, including the amplification of garden-path effects. Most critically, we demonstrate that resampling, a common practice with these models, inherently produces real-time digging-in effects -- where disambiguation difficulty increases with ambiguous region length. Digging-in magnitude scales inversely with particle count: fully parallel models predict no such effect.

Related papers

• Surprisal Theory is Tautological (without Rational Grounding) [60.917028769172795]
  Surprisal theory holds that the human processing difficulty of a linguistic unit in context is an affine function of its surprisal under some language model.<n>I argue this claim is a tautology without further constraint.<n>I conclude that breaking the tautology requires a rationalist intervention.
  arXiv  Detail & Related papers  (2026-07-23T17:54:37Z)
• Trajectory Dynamics in Language Model Hidden States Predict Human Processing Costs Beyond Surprisal [0.0]
  We show that word-level prediction error (surprisal) and sensitivity to the local momentum of the unfolding interpretation (trajectory extrapolation error) are dissociable components of processing cost.<n>The effect is especially pronounced in garden-path sentences.
  arXiv  Detail & Related papers  (2026-06-03T18:44:41Z)
• Why are language models less surprised than humans? Testing the Parse Multiplicity Mismatch Hypothesis [14.478855009044807]
  Surprisal theory posits that the processing difficulty of a word is determined by its predictability in context.<n> LMs successfully predict reading times in naturalistic text, but systematically underpredict the magnitude of difficulty observed in controlled studies of syntactic ambiguity.<n>This suggests that differences in the number of simultaneous parses available to LMs and humans cannot reconcile LM-based surprisal with human sentence processing.
  arXiv  Detail & Related papers  (2026-05-14T21:44:26Z)
• Revisiting Real-Time Digging-In Effects: No Evidence from NP/Z Garden-Paths [10.217186322563318]
  We report experiments on English NP/Z garden-path sentences using Maze and self-paced reading.<n>We find no evidence for real-time digging-in effects.
  arXiv  Detail & Related papers  (2026-03-24T18:07:46Z)
• Hybrid Neural-LLM Pipeline for Morphological Glossing in Endangered Language Documentation: A Case Study of Jungar Tuvan [6.367163817135528]
  We present a hybrid automatic glossing pipeline that combines neural sequence labeling with large language model (LLM) post-correction.<n>We show that retrieval-augmented prompting provides substantial gains over random example selection.<n>We also find that morpheme dictionaries paradoxically hurt performance compared to providing no dictionary at all in most cases.
  arXiv  Detail & Related papers  (2026-03-01T05:03:11Z)
• Towards Long Context Hallucination Detection [49.195854802543714]
  Large Language Models (LLMs) have demonstrated remarkable performance across various tasks.<n>They are prone to contextual hallucination, generating information that is either unsubstantiated or contradictory to the given context.<n>We propose a novel architecture that enables pre-trained encoder models, such as BERT, to process long contexts and effectively detect contextual hallucinations.
  arXiv  Detail & Related papers  (2025-04-28T03:47:05Z)
• The Impact of Token Granularity on the Predictive Power of Language Model Surprisal [15.073507986272027]
  One factor that has been overlooked in cognitive modeling is the granularity of subword tokens.<n>Experiments with naturalistic reading times reveal a substantial influence of token granularity on surprisal.<n>On garden-path constructions, language models trained on coarser-grained tokens generally assigned higher surprisal to critical regions.
  arXiv  Detail & Related papers  (2024-12-16T16:24:58Z)
• Gumbel Counterfactual Generation From Language Models [64.55296662926919]
  We show that counterfactual reasoning is conceptually distinct from interventions.<n>We propose a framework for generating true string counterfactuals.<n>We show that the approach produces meaningful counterfactuals while at the same time showing that commonly used intervention techniques have considerable undesired side effects.
  arXiv  Detail & Related papers  (2024-11-11T17:57:30Z)
• SHINE: Saliency-aware HIerarchical NEgative Ranking for Compositional Temporal Grounding [52.98133831401225]
  Temporal grounding, also known as video moment retrieval, aims at locating video segments corresponding to a given query sentence. We propose a large language model-driven method for negative query construction, utilizing GPT-3.5-Turbo. We introduce a coarse-to-fine saliency ranking strategy, which encourages the model to learn the multi-granularity semantic relationships between videos and hierarchical negative queries.
  arXiv  Detail & Related papers  (2024-07-06T16:08:17Z)
• Do Language Models Exhibit Human-like Structural Priming Effects? [3.4435563735186747]
  We investigate which linguistic factors play an important role in influencing language model predictions. We make use of the structural priming paradigm, where recent exposure to a structure facilitates processing of the same structure. We show that these effects can be explained via the inverse frequency effect, known in human priming, where rarer elements within a prime increase priming effects.
  arXiv  Detail & Related papers  (2024-06-07T11:21:52Z)
• Are Representations Built from the Ground Up? An Empirical Examination of Local Composition in Language Models [91.3755431537592]
  Representing compositional and non-compositional phrases is critical for language understanding. We first formulate a problem of predicting the LM-internal representations of longer phrases given those of their constituents. While we would expect the predictive accuracy to correlate with human judgments of semantic compositionality, we find this is largely not the case.
  arXiv  Detail & Related papers  (2022-10-07T14:21:30Z)
• When is Particle Filtering Efficient for Planning in Partially Observed Linear Dynamical Systems? [60.703816720093016]
  This paper initiates a study on the efficiency of particle filtering for sequential planning. We are able to bound the number of particles needed so that the long-run reward of the policy based on particle filtering is close to that based on exact inference. We believe this technique can be useful in other sequential decision-making problems.
  arXiv  Detail & Related papers  (2020-06-10T17:43:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.