Beyond the Trace: Coupling an Interpretable Reasoning-State Readout to Native MoE Routing

• URL: http://arxiv.org/abs/2608.17638v1
• Date: Tue, 18 Aug 2026 10:56:31 GMT
• Title: Beyond the Trace: Coupling an Interpretable Reasoning-State Readout to Native MoE Routing
• Abstract summary: We introduce a two-level internal readout for mixture-of-experts reasoning.<n>We first distill vocabulary-scale J-space into J64, a 64-axis semantic frame learned from the model's own reasoning states.<n>We then reconstruct J64 from native expert-routing statistics.
• Score: 61.695980784491475
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: What a reasoning model writes is only a partial record of the process that produces it. We introduce a two-level internal readout for mixture-of-experts reasoning. We first distill vocabulary-scale J-space into J64, a 64-axis semantic frame learned from the model's own reasoning states. J64 reveals readable process state that the emitted trace does not show: it separates inference effort from problem-induced strain. It also adds 0.096 to 0.135 held-out AUC over a baseline that reads the same rollout as token occupancy and aggregates it in exactly the same way. We then reconstruct J64 from native expert-routing statistics. The result is R64, a low-overhead proxy: its median per-axis correlation with J64 is 0.69 to 0.86 across three models and two families, and on gpt-oss-20b it preserves 95 to 100% of J64's predictive gain. The readout supports test-time decisions at two temporal resolutions. Over completed candidate sets, J64 and R64 improve single-branch selection, and R64-weighted voting improves plain majority voting in seven of eight settings. During generation, rolling readout windows drive a cumulative stop-and-resample policy whose operating point is fixed on training questions alone. J64 improves accuracy by 1.1 to 5.9 points over a sibling-permuted control, and the routing-only R64 proxy retains 0.9 to 3.2 of those points. Finally, router edits aimed at the mechanism J64 names induce the predicted reasoning behaviors and shift a diagnosed stall from numerical guessing toward exact symbolic execution. Together, J64 makes latent process state readable, while routing makes it deployable and actionable.

Related papers

• Interface-Induced Trajectory Censoring [6.496285418517832]
  We report a tool-call rate read off the serving stack.<n>That number can be zero while the model is emitting well-formed calls.<n>We reproduce the funnel across a 21x scale range of Qwen2.5-Coder.
  arXiv  Detail & Related papers  (2026-09-03T15:00:31Z)
• Disagree to Explore, Agree to Commit: Routing-Guided Test-Time Scaling for Software Agents [60.650808962786364]
  We study whether native MoE router traces can guide steering and selection without an external judge or selection-time test execution.<n>Our analysis shows that routing provides a robust behavioral role signal; token-granular readouts and decision-matched comparison sets turn it into effective control.<n>We evaluate on SWE-bench Verified using open-weight sparse MoE agents across scales and reasoning-effort settings.
  arXiv  Detail & Related papers  (2026-08-23T03:07:03Z)
• The Recall Trap: A Recall-Maximizing Retriever Configuration Reduces Issue Resolution in Fixed-Budget Code Context [45.88028371034407]
  We report a controlled case study in code repair, not a new phenomenon.<n>We toggle one flag (one-chunk-perfile deduplication) on an otherwise identical stack.<n>We map where it holds: it reverses on a lexical BM25 (3.2pp, significant cross-paradigm interaction)<n>We show that at a tight fixed budget, do not hard-deduplicate by file, and A/B packing policies against the task, not the metric flag was tuned.
  arXiv  Detail & Related papers  (2026-08-14T19:22:50Z)
• QuoteBench: How Matched Scores Can Hide Command-Path Failures [30.480838412827907]
  Execution scores alone cannot distinguish command-generation errors from failures introduced after generation.<n> QuoteBench measures this boundary with exact final-state validation on 56 one-shot tasks from 14 incident-derived families.<n> Evaluations of command-issuing agents should report the model configuration, generation contract, execution path, operating point, and final-state rather than treat a matched score as an intrinsic model property.
  arXiv  Detail & Related papers  (2026-08-13T17:57:20Z)
• Ready Cohorts: Bounding GPU Opportunity and Avoiding Host Round Trips in LLM-Agent Control [0.0]
  We formalize the ready-cohort boundary using fixed-share F, exact offline share P*, local upper bound U, and online achieved share A.<n>A separate mechanism study keeps a GPU-computed binary decision on device instead of returning four bytes to the host and redispatching.
  arXiv  Detail & Related papers  (2026-08-12T14:42:15Z)
• Policy-Masked Private Experts: Auditable and Reversible Capability Access Control in Sparse MoE Models [0.0]
  Policy-Masked Private Experts freezes a pretrained sparse Mixture-of-Experts (MoE) model, trains a disjoint expert branch, and selects the public or private pool before top-k routing.<n>We test this separation between execution control and task utility in Qwen3-30B-A3B and DeepSeek-V2-Lite.
  arXiv  Detail & Related papers  (2026-08-07T01:39:36Z)
• AgentDebugX: An Open-Source Toolkit for Failure Observability, Attribution, and Recovery in LLM Agents [83.37969790806069]
  LLM agent failures are difficult to debug because the step where an error surfaces is often not the one that caused it.<n>We present Agent DebuggerX, an open-source framework that organizes debug as a closed loop of Detect, Attribute, Recover, and Rerun.<n>At its core, Deep Debugger performs multi-turn root-cause diagnosis through global trajectory understanding.
  arXiv  Detail & Related papers  (2026-07-21T06:21:13Z)
• Operational Proto-Introspection in Looped Language Models: Process-Quality Taps, Executable Branching, and the Readout-Control Boundary [0.0]
  Can a language model read the quality of its ongoing computation?<n>Can an external intervention turn that readout into better outcomes?<n>We test both questions in a frozen 2.6B looped transformer, Ouro-RLTT.
  arXiv  Detail & Related papers  (2026-07-20T22:40:36Z)
• Does the Same Token Mean the Same State? MoE Routing as Signal for Reasoning Control [53.43997385459622]
  We operationalize textbfRAD (Routing Agreement Decoding) as an answer-string-free multi-rollout selector.<n>It locates a fixed anchor, represents each rollout by its anchor-window MoE routing states, and returns the densest Weighted-Jaccard $K$-NN route-basin center.
  arXiv  Detail & Related papers  (2026-06-22T03:17:31Z)
• RepoGenesis: Benchmarking End-to-End Microservice Generation from Readme to Repository [52.98970048197381]
  RepoGenesis is the first multilingual benchmark for repository-level end-to-end web microservice generation.<n>It consists of 106 repositories (60 Python, 46 Java) across 18 domains and 11 frameworks, with 1,258 API endpoints and 2,335 test cases verified.<n>Results reveal that despite high AC (up to 73.91%) and DSR (up to 100%), the best-performing system achieves only 23.67% Pass@1 on Python and 21.45% on Java.
  arXiv  Detail & Related papers  (2026-01-20T13:19:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.