The Complexity Ceiling Benchmark: A Multi-Domain Evaluation of Sequential Reasoning Under Depth Scaling

• URL: http://arxiv.org/abs/2606.29278v1
• Date: Sun, 28 Jun 2026 08:56:34 GMT
• Title: The Complexity Ceiling Benchmark: A Multi-Domain Evaluation of Sequential Reasoning Under Depth Scaling
• Abstract summary: We introduce the Complexity Ceiling Benchmark (CCB), a controlled evaluation of how language-model reasoning decays.<n>CCB fixes the semantic content of a task and varies only its depth N in 5,...,50 across three structurally distinct regimes.<n>We find a consistent pattern of geometric per-step decay with widely separated domain ceilings.
• Score: 8.616356693448987
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce the Complexity Ceiling Benchmark (CCB), a controlled evaluation of how language-model reasoning decays as the number of required sequential steps grows. CCB fixes the semantic content of a task and varies only its depth N in {5,...,50} across three structurally distinct regimes: grounded spatial state-tracking, abstract symbolic pointer manipulation, and transitive relational inference. Across 6,000 trials over five frontier and open-weight LLMs we find a consistent pattern of geometric per-step decay with widely separated domain ceilings: on the first two regimes the strongest models retain pd>0.92 across N=50; on the third every model collapses by N=5, with the best model's 50%-success horizon at H0.5~4.7 steps despite pd=0.863. A trace-level metric (TFBC) shows that 14.5% of correct answers across the benchmark are reached via incorrect intermediate reasoning. Forced verbose state-tracking does not move the ceiling (McNemar p=1.000), and the mean step at which reasoning first diverges, k*, predicts within-domain accuracy better than parameter count. CCB and the geometric decay model together reduce a model's long-horizon reasoning profile to one interpretable number per task family.

Related papers

• A*-Thought-V2: Efficient Latent Reasoning via Geometric Dynamics of LLM [40.07171947823021]
  Chain-of-Thought (CoT) improves the reasoning ability of Large Language Models (LLMs) but incurs substantial computation and context costs.<n>We present A*-Thought-V2, a geometric dynamics of LLM guided framework that models CoT as a hidden-state trajectory and replaces hard deletion with an explicit-implicit interleaved latent architecture.
  arXiv  Detail & Related papers  (2026-09-07T17:56:20Z)
• Architecture-Dependent Causal Transfer of Activation States Across Large Language Models [51.56484100374058]
  Internal language activation states can be transferred causally between different large language models.<n>Injecting projected activations into a target model during generation produces a statistically significant, preregistered causal effect.<n>We conclude that end-to-end activation-state transfer between LLMs, as currently implemented, is architecture-dependent rather than universal.
  arXiv  Detail & Related papers  (2026-08-17T09:53:27Z)
• From Direction to Magnitude: How Multimodal Instruction-Tuning Reorganizes the Geometric Encoding of Identity-Specifying Prompts in Transformer Hidden States [0.0]
  We investigate whether identity-specifying system prompts produce statistically distinguishable geometric fingerprints in the hidden-state trajectories of four open-weight transformer language models.<n>Three prompt conditions are compared via five metrics, principally the 1-Wasserstein distance between edge-wise of Ollivier-Ricci curvature on kNN trajectory graphs.
  arXiv  Detail & Related papers  (2026-07-10T17:22:36Z)
• Automated Proving of Shannon-Type Entropy Inequalities via Fine-Tuned Language Models and Guided Tree Search [50.16356451328644]
  Proving Shannon-type entropy inequalities is a fundamental task in information theory.<n>We investigate whether small-scale large language models, fine-tuned on atomic proof steps, can automate this process.<n>We show that GPT-5.5 solves 1.7% samples under zero-shot prompting while Psitip solves 33.3% samples.
  arXiv  Detail & Related papers  (2026-06-04T05:43:12Z)
• Right Makes Might: Aligning Verified Hidden States Empowers RL Reasoning [55.264863369127774]
  Current methods reduce each correct rollout to a single reward bit, ignoring the geometric structure shared among their hidden states.<n>We propose Hidden-Align, an auxiliary loss function that aligns the last-layer hidden states of correct rollouts at the anchor token during RL training, with zero overhead in both training and inference.<n>On eight mathematical reasoning benchmarks, Hidden-Align improves average pass@1 over the DAPO baseline by 3.8, 6.2, and 5.4 percentage points on Qwen3-1.7B, 4B, and 14B respectively, with consistent pass@k gains across all three scales.
  arXiv  Detail & Related papers  (2026-06-02T06:51:15Z)
• Convergence Theory for Iterative LLM-Based Neural Architecture Search: A Parametric Cross-Entropy Framework with Closed-Form Proxy Reliability [48.83701310501069]
  Large language models (LLMs) are increasingly used as generators in iterative neural architecture search (NAS)<n>We model iterative LLM-NAS as a parametric Cross-Entropy (CE) method over executable programs.<n>We prove six results: (1) iterative LLM fine-tuning on elite architectures is equivalent to the CE update restricted to the LLM family; (2) expected architecture quality is monotonically non-decreasing across cycles; and (3) elite-set probability converges to a fixed point at a geometric rate.
  arXiv  Detail & Related papers  (2026-05-28T15:45:19Z)
• Self-Training Doesn't Flatten Language -- It Restructures It: Surface Markers Amplify While Deep Syntax Dies [4.738949927143789]
  Successive self-training on a language model's own outputs is widely characterized as a process of flattening.<n>We provide evidence that this characterization is incomplete.<n>Across eleven generations of self-training on five models, language is not flattened uniformly -- it is restructured.
  arXiv  Detail & Related papers  (2026-05-20T01:44:47Z)
• Looped SSMs: Depth-Recurrence and Input Reshaping for Time Series Classification [50.994194925685434]
  We show that a looped SSM with $k$ parameters iterated $L$ times consistently closely matches or outperforms a standard SSM.<n>We also show that input reshaping is an equally neglected design axis.
  arXiv  Detail & Related papers  (2026-05-15T15:18:12Z)
• The Evaluation Blind Spot: A Stereological Theory of Benchmark Coverage for Large Language Models [0.0]
  Empirically, three independent leaderboards (Open LLM v2, an extended 12-benchmark suite, LiveBench) all have d_eff in [2.86, 4.80] on their competitive frontier.<n>A submodular greedy algorithm with the Nemhauser (1 - 1/e) guarantee finds a stable core of 4 benchmarks.<n>A counterfactual validation across 12 internal benchmarks and 27 categories confirms that the eigenstructure predicts which evaluations are irreplaceable.
  arXiv  Detail & Related papers  (2026-04-15T08:56:58Z)
• When Self-Reference Fails to Close: Matrix-Level Dynamics in Large Language Models [0.0]
  Self-referential statements and meta-cognitive prompts are more stable than paradoxical self-reference on key collapse-related metrics.<n>We propose, as a conjecture, that NCTR forces finite-depth transformers toward dynamical regimes where these problems concentrate.
  arXiv  Detail & Related papers  (2026-04-13T23:23:02Z)
• On the Role of Reasoning Patterns in the Generalization Discrepancy of Long Chain-of-Thought Supervised Fine-Tuning [63.41902113656453]
  Supervised Fine-Tuning (SFT) on long Chain-of-Thought (CoT) trajectories has become a pivotal phase in building large reasoning models.<n>We conduct a comparative study using two sources of verified CoT trajectories generated by two competing models.<n>We find that SFT on textttDeepSeek-R1-0528 data achieves remarkably lower training loss, yet exhibits significantly worse generalization performance.
  arXiv  Detail & Related papers  (2026-04-02T07:00:54Z)
• Complexity Agnostic Recursive Decomposition of Thoughts [3.3864434164156934]
  We introduce CARD (Complexity Agnostic Recursive Decomposition), a framework that predicts problem complexity before generation and adapts decomposition accordingly.<n> CARD achieves 81.4% to 89.2% accuracy on GSM8K while reducing token cost by 1.88x to 2.40x compared to fixed decomposition baselines.
  arXiv  Detail & Related papers  (2025-12-10T06:03:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.