Prediction Bottlenecks Don't Discover Causal Structure (But Here's What They Actually Do)

• URL: http://arxiv.org/abs/2605.09169v1
• Date: Sat, 09 May 2026 21:12:55 GMT
• Title: Prediction Bottlenecks Don't Discover Causal Structure (But Here's What They Actually Do)
• Abstract summary: A Mamba state-space model trained only for next-step prediction appears to recover Granger-causal structure through a simple readout.<n>We package the protocol used to test that claim as a reusable falsification benchmark, and walk the claim through it in five stages.<n>What survives is a narrow characterization result; the benchmark is the lasting artifact, and each stage above is one of its control arms.
• Score: 11.664243197168403
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A Mamba state-space model trained only for next-step prediction appears to recover Granger-causal structure through a simple readout $S = |W_{out} W_{in}|$, with early experiments suggesting the phenomenon generalized across architectures and benefited from interventional data at $p < 10^{-5}$. We package the protocol used to test that claim -- standardized synthetic generators (VAR/Lorenz/CauseMe-style), three intervention semantics ($do(X=c)$, soft-noise, random-forcing), edge-provenance cards on three real datasets, and size-matched control arms -- as a reusable falsification benchmark, and walk the claim through it in five stages. The method-level claim does not survive: (i) a plain linear bottleneck does as well or better; (ii) tuned Lasso beats the bottleneck on synthetic CauseMe-style benchmarks, and on Lorenz-96 (the only real benchmark with unambiguous ground truth) classical PCMCI and Granger lead a tight cluster in which the bottleneck trails; (iii) the headline intervention advantage is roughly 60% a sample-size confound, and the residual disappears under standard $do(X=c)$ interventions, surviving only under a non-standard random-forcing scheme; (iv) even that residual reproduces, with a larger effect, in classical bivariate Granger -- the effect is method-agnostic. What survives is a narrow characterization result; the benchmark is the lasting artifact, and each stage above is one of its control arms.

Related papers

• Architecture-Dependent Causal Transfer of Activation States Across Large Language Models [51.56484100374058]
  Internal language activation states can be transferred causally between different large language models.<n>Injecting projected activations into a target model during generation produces a statistically significant, preregistered causal effect.<n>We conclude that end-to-end activation-state transfer between LLMs, as currently implemented, is architecture-dependent rather than universal.
  arXiv  Detail & Related papers  (2026-08-17T09:53:27Z)
• Adaptive operator-generated subspaces for effective many-body Hamiltonians [0.0]
  We present the Adaptive Clifford-Algebra Subspacesolver (A-CASE)<n>Overlap, Hamiltonian, observable, and response matrices are reconstructed from one shared Pauli-expectation bank.<n>A grouped iteration propagates finite-shot variability through thresholding, root matching, spectral weights, susceptibility, and broadening.
  arXiv  Detail & Related papers  (2026-08-01T09:51:25Z)
• Modeling Normal Is All You Need: Joint Latent Clustering for Anomaly Detection in Multimodal Cyber-Physical Systems [47.027290803102666]
  Normal behaviour is the union of many imbalanced, curved, thin-fringed operating regimes.<n>We model the normal law with a jointly learned latent representation plus explicit Gaussian-mixture mode clustering.<n>We evaluate under a deliberately fair protocol: raw point-wise metrics with no point adjustment, a trivial-detector difficulty split, prevalence-matched F1, and train-normal-only calibration.
  arXiv  Detail & Related papers  (2026-07-07T10:10:00Z)
• GRACE: Gated Refinement for Accurate Causal Edge Discovery in High-Dimensional Time Series [0.0]
  Fast linear CI skeleton provides high-recall candidates.<n>A single gated model prunes false positives by learning which edges genuinely improve prediction.<n> GRACE matches or exceeds expensive nonlinear CI tests at a fraction of the cost.
  arXiv  Detail & Related papers  (2026-06-22T19:26:38Z)
• Convergence Theory for Iterative LLM-Based Neural Architecture Search: A Parametric Cross-Entropy Framework with Closed-Form Proxy Reliability [48.83701310501069]
  Large language models (LLMs) are increasingly used as generators in iterative neural architecture search (NAS)<n>We model iterative LLM-NAS as a parametric Cross-Entropy (CE) method over executable programs.<n>We prove six results: (1) iterative LLM fine-tuning on elite architectures is equivalent to the CE update restricted to the LLM family; (2) expected architecture quality is monotonically non-decreasing across cycles; and (3) elite-set probability converges to a fixed point at a geometric rate.
  arXiv  Detail & Related papers  (2026-05-28T15:45:19Z)
• The Surprising Universality of LLM Outputs: A Real-Time Verification Primitive [0.0]
  CPU-only scoring primitive runs at 2.6 microseconds per token.<n> token rank-frequency distributions converge to the same two- parameter Mandelbrot ranking distribution.<n>We derive a single-pass scoring primitive that composes with model log probabilities when available and degrades to a rank-only mode usable on closed APIs.
  arXiv  Detail & Related papers  (2026-04-28T13:35:31Z)
• Score-Repellent Monte Carlo: Toward Efficient Non-Markovian Sampler with Constant Memory in General State Spaces [48.37754141688874]
  We propose a framework that summarizes history by a running average of score evaluations in $Rd$, where $d$ is the dimension of the score and state representation.<n>This history is converted into a surrogate target through an exponential score tilt, indexed with $$ that represents the strength of repellence in controlling the magnitude of the history-based repulsion.<n>Experiments on continuous targets and discrete energy-based models demonstrate improved estimator variance and mode coverage, while retaining $O(d)$ memory usage and modest per-it overhead.
  arXiv  Detail & Related papers  (2026-04-24T18:39:50Z)
• Benchmarking the Utility of Privacy-Preserving Cox Regression Under Data-Driven Clipping Bounds: A Multi-Dataset Simulation Study [0.0]
  Differential privacy (DP) is a mathematical framework that guarantees individual privacy.<n>This study systematically evaluated the impact of DP mechanisms with data-driven clipping bounds on the Cox proportional hazards model.
  arXiv  Detail & Related papers  (2026-04-23T09:53:15Z)
• One Token Away from Collapse: The Fragility of Instruction-Tuned Helpfulness [12.183451602438753]
  We show that simple lexical constraints (banning a single punctuation character or common word) cause instruction-tuned LLMs to collapse their responses.<n>We show that base models show no systematic collapse under identical constraints, with effects that are small, noisy, and bidirectional.
  arXiv  Detail & Related papers  (2026-04-14T17:40:01Z)
• Synthetic Tabular Generators Fail to Preserve Behavioral Fraud Patterns: A Benchmark on Temporal, Velocity, and Multi-Account Signals [0.0]
  We formalize a taxonomy of four behavioral fraud patterns (P1-P4) covering inter-event timing, burst structure, multi-account graph motifs, and velocity-rule trigger rates.<n>We benchmark CTGAN, TVAE, GaussianCopula, and TabularARGN on IEEE-CIS Fraud Detection and the Amazon Fraud dataset.<n>The P1-P4 framework extends to any domain with entity-level sequential data, including healthcare and network security.
  arXiv  Detail & Related papers  (2026-04-13T19:36:00Z)
• Phase-space entropy at acquisition reflects downstream learnability [54.4100065023873]
  We propose an acquisition-level scalar $S_mathcal B$ based on instrument-resolved phase space.<n>We show theoretically that (S_mathcal B) correctly identifies the phase-space coherence of periodic sampling.<n>$|S_mathcal B|$ consistently ranks sampling geometries and predicts downstream reconstruction/recognition difficulty emphwithout training.
  arXiv  Detail & Related papers  (2025-12-22T10:03:51Z)
• Skewness-Robust Causal Discovery in Location-Scale Noise Models [47.09233752567902]
  We propose SkewD, a likelihood-based algorithm for causal discovery under location-scale noise models.<n>SkewD extends the usual normal-distribution framework to the skew-normal setting, enabling reliable inference under symmetric and skewed noise.<n>We evaluate SkewD on novel synthetically generated datasets with skewed noise as well as established benchmark datasets.
  arXiv  Detail & Related papers  (2025-11-18T12:40:41Z)
• Latency and Ordering Effects in Online Decisions [0.0]
  Online decision systems operate under delayed feedback and order-sensitive dynamics.<n>We package heterogeneous latency, noncommutativity, and implementation-gap effects into a single lower-bound statement.
  arXiv  Detail & Related papers  (2025-11-17T07:08:05Z)
• Computational-Statistical Tradeoffs at the Next-Token Prediction Barrier: Autoregressive and Imitation Learning under Misspecification [50.717692060500696]
  Next-token prediction with the logarithmic loss is a cornerstone of autoregressive sequence modeling.<n>Next-token prediction can be made robust so as to achieve $C=tilde O(H)$, representing moderate error amplification.<n>No computationally efficient algorithm can achieve sub-polynomial approximation factor $C=e(log H)1-Omega(1)$.
  arXiv  Detail & Related papers  (2025-02-18T02:52:00Z)
• Identifying General Mechanism Shifts in Linear Causal Representations [58.6238439611389]
  We consider the linear causal representation learning setting where we observe a linear mixing of $d$ unknown latent factors. Recent work has shown that it is possible to recover the latent factors as well as the underlying structural causal model over them. We provide a surprising identifiability result that it is indeed possible, under some very mild standard assumptions, to identify the set of shifted nodes.
  arXiv  Detail & Related papers  (2024-10-31T15:56:50Z)
• Almost Tight L0-norm Certified Robustness of Top-k Predictions against Adversarial Perturbations [78.23408201652984]
  Top-k predictions are used in many real-world applications such as machine learning as a service, recommender systems, and web searches. Our work is based on randomized smoothing, which builds a provably robust classifier via randomizing an input. For instance, our method can build a classifier that achieves a certified top-3 accuracy of 69.2% on ImageNet when an attacker can arbitrarily perturb 5 pixels of a testing image.
  arXiv  Detail & Related papers  (2020-11-15T21:34:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.