Hallucination as Trajectory Commitment: Causal Evidence for Asymmetric Attractor Dynamics in Transformer Generation

• URL: http://arxiv.org/abs/2604.15400v1
• Date: Thu, 16 Apr 2026 12:16:53 GMT
• Title: Hallucination as Trajectory Commitment: Causal Evidence for Asymmetric Attractor Dynamics in Transformer Generation
• Abstract summary: hallucination in autoregressive language models is governed by asymmetric attractor dynamics.<n>We use same-prompt bifurcation to isolate trajectory dynamics from prompt-level confounds.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present causal evidence that hallucination in autoregressive language models is an early trajectory commitment governed by asymmetric attractor dynamics. Using same-prompt bifurcation, in which we repeatedly sample identical inputs to observe spontaneous divergence, we isolate trajectory dynamics from prompt-level confounds. On Qwen2.5-1.5B across 61 prompts spanning six categories, 27 prompts (44.3%) bifurcate with factual and hallucinated trajectories diverging at the first generated token (KL = 0 at step 0, KL > 1.0 at step 1). Activation patching across 28 layers reveals a pronounced causal asymmetry: injecting a hallucinated activation into a correct trajectory corrupts output in 87.5% of trials (layer 20), while the reverse recovers only 33.3% (layer 24); both exceed the 10.4% baseline (p = 0.025) and 12.5% random-patch control. Window patching shows correction requires sustained multi-step intervention, whereas corruption needs only a single perturbation. Probing the prompt encoding itself, step-0 residual states predict per-prompt hallucination rate at Pearson r = 0.776 at layer 15 (p < 0.001 against a 1000-permutation null); unsupervised clustering identifies five regime-like groups (eta^2 = 0.55) whose saddle-adjacent cluster concentrates 12 of the 13 bifurcating false-premise prompts, indicating that the basin structure is organized around regime commitments fixed at prompt encoding. These findings characterize hallucination as a locally stable attractor basin: entry is probabilistic and rapid, exit demands coordinated intervention across layers and steps, and the relevant basins are selected by clusterable regimes already discernible at step 0.

Related papers

• K/V-Cache Interventions Dissociate Representation Alignment from Persona Expression in Decoder-Only Language Models [32.7114174898989]
  We study K/V-cache interventions as a structured surface for persona control in decoder-only language models.<n>We report two consistent universalitys between representation-level alignment and behavioral expression, plus a common failure under position perturbations.
  arXiv  Detail & Related papers  (2026-09-10T02:56:50Z)
• Lagged Coupling: Internal Representations Become Readable Before They Become Causal [0.0]
  Internal readability systematically outruns causal efficacy, and the lag does not shrink with scale.<n>A pre-registered OLMo-2 replication preserves the direction at attenuated magnitude.
  arXiv  Detail & Related papers  (2026-09-01T10:46:42Z)
• Evidence-Type Competition: When Can Interventional Data Teach Language Models Causal Direction? [0.0]
  interventional data is widely regarded as the gold standard for teaching models causal reasoning.<n>We test this assumption in a fully controlled synthetic environment pitting observational correlation against causal effect.<n>What governs whether interventional evidence is used is not the training mixture but the evidence type present in the context at inference time.
  arXiv  Detail & Related papers  (2026-07-31T14:49:18Z)
• Verifier-Guided Twelve-Tone Composition: A Generate-Verify-Repair Harness for Symbolic Music Generation [37.048771679940984]
  We introduce a neuro-symbolic harness that wraps a language-model proposer in a generate-verify-repair-trace loop with symbolic verification.<n>The complete pipeline improves event-local consistency without claiming whole-piece legality.
  arXiv  Detail & Related papers  (2026-07-13T09:52:09Z)
• Scaffold splits hide structural-frontier failures in ADMET models [25.408600489964726]
  We introduce a label-free structural-frontier split that reserves the sparsest and most physicochemically remote scaffold groups.<n>We evaluate it on six public experimental or curated ADMET tasks.
  arXiv  Detail & Related papers  (2026-07-12T12:25:50Z)
• Distributional Energy-Based Models for Uncertainty-Aware Structured LLM Reasoning [40.342912574072024]
  Large Language Models produce structured outputs such as travel plans or code solutions.<n>Individual reasoning steps may appear correct while the output as a whole violates budgets, fails test cases, or contradicts earlier deductions.<n>We propose a learned quality scorer with deterministic analytical constraint penalties for verifying structured LLM outputs.
  arXiv  Detail & Related papers  (2026-05-15T17:08:27Z)
• Margin-calibrated Classifier Guidance for Property-driven Synthesis Planning [23.782236689125995]
  We introduce Sequence Completion Ranking (SCR), which employs contrastive argumentation and a margin-based loss to calibrate the classifier.<n>SCR substantially improves multi-step solve rates from $16.8%$ (unguided generator) to $78.4%$ with reaction-type guidance and $95.3%$ with Tanimoto guidance.<n>Our method also effectively closes the long-standing diversity gap between templatefree and template-based methods.
  arXiv  Detail & Related papers  (2026-05-13T07:12:53Z)
• Perturbation Probing: A Two-Pass-per-Prompt Diagnostic for FFN Behavioral Circuits in Aligned LLMs [9.127363793428119]
  Perturbation probing generates task-specific causal hypotheses for FFN neurons in large language models.<n>Across eight behavioral circuits, 13 models, and four architecture families, we identify two circuit structures that organize LLM behavior.
  arXiv  Detail & Related papers  (2026-04-30T04:13:33Z)
• The Autocorrelation Blind Spot: Why 42% of Turn-Level Findings in LLM Conversation Analysis May Be Spurious [35.76482964927589]
  We characterize the autocorrelation structure of 66 turn-level metrics across 202 multi-turn conversations.<n>We show that 42% of associations that appear significant under standard pooled testing fail to survive cluster-robust correction.<n>We present a two-stage correction framework combining Chelton (1983) effective degrees of freedom with conversation-level block bootstraps.
  arXiv  Detail & Related papers  (2026-04-15T20:54:39Z)
• Hierarchical Kernel Transformer: Multi-Scale Attention with an Information-Theoretic Approximation Analysis [0.0]
  The Hierarchical Kernel Transformer (HKT) is a multi-scale attention mechanism that processes sequences at L resolution levels via trainable causal downsampling.<n>The total computational cost is bounded by 4/3 times that of standard attention, reaching 1.3125x for L = 3.
  arXiv  Detail & Related papers  (2026-04-10T00:00:11Z)
• Before the First Token: Scale-Dependent Emergence of Hallucination Signals in Autoregressive Language Models [6.5607014897534865]
  We study the temporal dynamics of hallucination-indicative internal representations across 7 autoregressive transformers.<n>We identify a scale-dependent phase transition: models below 400M parameters show chance-level probe accuracy at every generation position.<n>At the 7B scale, we observe a striking dissociation: Pythia-6.9B produces a flat temporal profile, while instruction-tuned Qwen2.5-7B shows a dominant pre-generation effect.
  arXiv  Detail & Related papers  (2026-03-20T02:30:01Z)
• Exhaustive Circuit Mapping of a Single-Cell Foundation Model Reveals Massive Redundancy, Heavy-Tailed Hub Architecture, and Layer-Dependent Differentiation Control [0.0]
  We present three experiments that address limitations through exhaustive circuit tracing, higher order ablation, and causal trajectory steering.<n>First, exhaustive tracing of all 4065 active autoencoder features at layer 5 yields 1393850 significant downstream edges.<n>Second, three way redundancy across 8 feature triplets shows that redundancy deepens monotonically with interaction order.<n>Third, trajectory guided feature steering establishes a causal link between layer position and differentiation directionality.
  arXiv  Detail & Related papers  (2026-03-12T13:53:20Z)
• From Prerequisites to Predictions: Validating a Geometric Hallucination Taxonomy Through Controlled Induction [0.0]
  We test whether a geometric hallucination taxonomy can distinguish hallucination types.<n>Type1 and2 do not separate in either space.<n>Type1/2 non-separation is genuine at 124M parameters.
  arXiv  Detail & Related papers  (2026-02-27T20:56:59Z)
• BadCLIP++: Stealthy and Persistent Backdoors in Multimodal Contrastive Learning [73.46118996284888]
  Research on backdoor attacks against multimodal contrastive learning models faces two key challenges: stealthiness and persistence.<n>We propose BadCLIP++, a unified framework that tackles both challenges.<n>For stealthiness, we introduce a semantic-fusion QR micro-trigger that embeds imperceptible patterns near task-relevant regions.<n>For persistence, we stabilize trigger embeddings via radius shrinkage and centroid alignment.
  arXiv  Detail & Related papers  (2026-02-19T08:31:16Z)
• AgentHallu: Benchmarking Automated Hallucination Attribution of LLM-based Agents [30.66751974860931]
  hallucination detection in single-turn responses requires identifying which step causes the initial divergence.<n>We propose a new research task, automated hallucination attribution of LLM-based agents, aiming to identify the step responsible for the hallucination and explain why.<n>We introduce AgentHallu, a comprehensive benchmark with 693 high-quality trajectories spanning 7 agent frameworks and 5 domains.<n>The best-performing model achieves only 41.1% step localization accuracy, where tool-use hallucinations are the most challenging at just 11.6%.
  arXiv  Detail & Related papers  (2026-01-11T09:04:26Z)
• Towards a Science of Scaling Agent Systems [79.64446272302287]
  We formalize a definition for agent evaluation and characterize scaling laws as the interplay between agent quantity, coordination structure, modelic, and task properties.<n>We derive a predictive model using coordination metrics, that cross-validated R2=0, enabling prediction on unseen task domains.<n>We identify three effects: (1) a tool-coordination trade-off: under fixed computational budgets, tool-heavy tasks suffer disproportionately from multi-agent overhead, and (2) a capability saturation: coordination yields diminishing or negative returns once single-agent baselines exceed 45%.
  arXiv  Detail & Related papers  (2025-12-09T06:52:21Z)
• Causal Understanding by LLMs: The Role of Uncertainty [43.87879175532034]
  Recent papers show LLMs achieve near-random accuracy in causal relation classification.<n>We investigate whether pretraining exposure to causal examples improves causal understanding.
  arXiv  Detail & Related papers  (2025-09-24T13:06:35Z)
• Hallucination, Monofacts, and Miscalibration: An Empirical Investigation [2.3278261859840104]
  Hallucinated facts in large language models obey a statistical lower bound determined by the monofact rate.<n>We present the first empirical investigation of this three-way relationship in classical n-gram models and fine-tuned encoder-decoder Transformers.
  arXiv  Detail & Related papers  (2025-02-11T18:46:00Z)
• Clustered Switchback Designs for Experimentation Under Spatio-temporal Interference [44.644520116360106]
  We estimate the global average treatment effect (GATE), the difference between average outcomes having exposed all units at all times to treatment or to control.<n>We propose a clustered switchback design, where units are grouped into clusters and time steps are grouped into blocks.<n>We show that for graphs that admit good clustering, a truncated Horvitz-Thompson estimator achieves a $tilde O(1/NT)$ mean squared error (MSE)<n>Our results simultaneously generalize the results from citethu2022switchback,ugander2013graph and citetleung2022rate
  arXiv  Detail & Related papers  (2023-12-25T01:00:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.