Interaction Scaling: Grounding the Third Axis of Test-Time Compute

• URL: http://arxiv.org/abs/2607.11598v1
• Date: Mon, 13 Jul 2026 14:22:06 GMT
• Title: Interaction Scaling: Grounding the Third Axis of Test-Time Compute
• Abstract summary: We argue that a single variable governs this third axis, grounding, and that it must hold on both sides of the loop.<n>On hard coding tasks at a fixed token budget, reasoning-only and best-of-N sampling both plateau.<n>Our proposer-reviewer harness reaches a perfect 100% pass rate with no run-to-run variance.
• Score: 0.33451037881913753
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: There are two standard ways to spend more compute at test time: let a model reason longer, or sample more attempts and keep one. Both share a hidden limit: they are internal. Every extra token comes from the same frozen weights and the same prompt, so neither can tell the model anything it does not already know. We study a third way, interaction: the model proposes an artifact, an external instrument observes how it actually behaves, and the model revises. Each cycle imports a real observation, so interaction breaks through the ceiling the other two hit. We argue that a single variable governs this third axis, grounding, and that it must hold on both sides of the loop. The feedback that drives revision must come from an instrument that actually observes the flaw, and so must the metric that scores the result. On hard coding tasks at a fixed token budget, reasoning-only and best-of-N sampling both plateau (the latter even when an oracle picks the best sample), while every interaction strategy keeps improving; our proposer-reviewer harness reaches a perfect 100% pass rate with no run-to-run variance, and the gain holds across three model families. On rendered visual artifacts, the usual judge (a vision-language model, or VLM, reading a screenshot) rates 14 of 15 visibly broken figures "perfect," because the screenshot hides the flaws before the judge can see them. A tool that measures the real layout instead shows the loop removing 40-74% of defects across four modalities; and that same VLM, used as the reviewer, makes slide layouts worse where the measuring tool repairs them. Interaction scaling is real and distinct from reasoning and sampling, but only visible when both the feedback and the metric are grounded.

Related papers

• Calibrating Interpretability Instruments Before Trusting Their Verdicts [1.0829694003408499]
  Causal claims about large language model (LLM) internals rest on measurements.<n>These measurements fail in specific, diagnosable ways that return a plausible number instead of an error.<n>This note documents six such failures from a causal interpretability program on refusal and moral representation.
  arXiv  Detail & Related papers  (2026-09-13T19:37:02Z)
• Reading the Room: Implicit Confusion Encoding in Recurrent World Model States [0.0]
  We show a recurrent hidden state $h_t$ trained to reduce prediction error tracks its own confusion.<n>It is functionally distinct from ensemble disagreement, which flags new inputs, and reconstruction error, which flags bad predictions right now.
  arXiv  Detail & Related papers  (2026-08-21T19:32:38Z)
• The Low Frequency Trap: Video Language Models Fail at Simple Event Bookkeeping [70.07961857926377]
  We introduce trace-grounded parametric profiling for event counting in three controlled video tasks.<n>Across 2,190 videos, we vary event count N and frequency F while holding rendering fixed.<n>At an 80% reliability threshold, Gemini 3.6 Flash reliably counts persistent state transitions up to 12 events at 0.5 and 1.0 Hz, yet demonstrates no reliable positive-count region for transient blinking events.
  arXiv  Detail & Related papers  (2026-08-06T17:57:06Z)
• The Illusion of Visual Tool-Use: A Causal Audit of Thinking with Images [19.417504594148223]
  "thinking-with-images" operations often achieve only marginal or negative gains over direct inference at substantially higher token cost.<n>We ask whether the returned visual evidence causally affects the answer.<n>We use visual tool-use as a causal graph that separates observation-mediated paths from action-induced shortcuts.
  arXiv  Detail & Related papers  (2026-08-06T17:01:08Z)
• LLMs as a Jury: Cross-Model Consensus Can Outperform Process Reward Models for LLM Reasoning [6.241883798820154]
  We study cross-model consensus, the degree to which independently trained models, each solving the problem once, agree on a final answer.<n>We treat the panel as an LLM-jury, in which the structure of agreement, not any model's score of another, is the verification signal.<n>We make this precise with a parameter-free law, derived in closed form, that predicts consensus accuracy from three measured panel statistics to a mean absolute error of $0.03$.
  arXiv  Detail & Related papers  (2026-07-11T05:57:54Z)
• Beyond the Current Observation: Evaluating Multimodal Large Language Models in Controllable Non-Markov Games [69.57330692969543]
  We introduce RNG-Bench, a benchmark suite designed to isolate a base model's ability to reconstruct past observations.<n>RNG-Bench includes two complementary games: Matching Pairs, where card identities briefly revealed at specific locations must later be recalled, and 3D Maze, where egocentric views must be integrated into a spatial map.<n>The hardest configurations require contexts of roughly 128K tokens and 350 image inputs per episode, and remain far from saturated by frontier MLLMs.
  arXiv  Detail & Related papers  (2026-06-17T17:59:34Z)
• Looking Is Not Picking: An Attention-Segment Account of Tool-Selection Failures in LLM Agents [0.4855342112648282]
  LLM agents mis-call tools, and the natural guess is that the model failed to see the right tool in a crowded harness.<n>We show the opposite through a lens concurrent work sets aside -- the model's attention to labeled tool-definition segments.
  arXiv  Detail & Related papers  (2026-06-15T07:58:56Z)
• VisualFLIP: Do Predictions Depend on Task-Critical Visual Evidence in Multimodal Reasoning? [79.44137632338062]
  When a multimodal large language model answers a visual reasoning question correctly, is the prediction actually supported by the task-critical visual evidence?<n>We introduce VisualFLIP, a paired benchmark with 1,374 images arranged as same-question pairs across cardinality, attribute, spatial, and logic tasks.<n>We evaluate 24 MLLMs with pair accuracy, which requires solving both sides of a pair, and Collapse Rate (CR), which measures how often a model that solves at least one side repeats the same non-empty answer for both images.
  arXiv  Detail & Related papers  (2026-06-05T22:06:07Z)
• StemBind: When MLLMs Get Lost Between Rules and Instances in Abstract Visual Reasoning [13.74158437063828]
  Multimodal large language models (MLLMs) often know the rule but pick the wrong answer on visual reasoning tasks.<n>Existing benchmarks cannot detect this because they collapse perception, rule induction, and answer selection into a single right-or-wrong signal.<n>We introduce StemBind, a shared-stem diagnostic benchmark that probes the same visual stem with three aligned questions.
  arXiv  Detail & Related papers  (2026-05-29T03:20:05Z)
• Silent Commitment Failure in Instruction-Tuned Language Models: Evidence of Governability Divergence Across Architectures [0.0]
  We introduce governability -- the degree to which a model's errors are detectable before output commitment and correctable once detected.<n>We show benchmark accuracy does not predict governability, correction capacity varies independently of detection, and identical governance scaffolds produce opposite effects across models.<n>We propose a Detection and Correction Matrix classifying model-task combinations into four regimes: Governable, Monitor Only, Steer Blind, and Ungovernable.
  arXiv  Detail & Related papers  (2026-03-22T21:50:28Z)
• e5-omni: Explicit Cross-modal Alignment for Omni-modal Embeddings [91.3041144166326]
  We propose e5-omni, a lightweight explicit alignment recipe that adapts off-the-shelf vision-language models into robust omni-modal embedding models.<n>e5-omni combines modality-aware temperature calibration to align similarity scales, a controllable negative curriculum with debiasing to focus on confusing negatives, and batch whitening to better match cross-modal geometry.<n> Experiments on MMEB-V2 and AudioCaps show consistent gains over strong bi-modal and omni-modal baselines.
  arXiv  Detail & Related papers  (2026-01-07T07:39:40Z)
• Show Your Work with Confidence: Confidence Bands for Tuning Curves [51.12106543561089]
  tuning curves plot validation performance as a function of tuning effort. We present the first method to construct valid confidence bands for tuning curves. We validate our design with ablations, analyze the effect of sample size, and provide guidance on comparing models with our method.
  arXiv  Detail & Related papers  (2023-11-16T00:50:37Z)
• Warp Consistency for Unsupervised Learning of Dense Correspondences [116.56251250853488]
  Key challenge in learning dense correspondences is lack of ground-truth matches for real image pairs. We propose Warp Consistency, an unsupervised learning objective for dense correspondence regression. Our approach sets a new state-of-the-art on several challenging benchmarks, including MegaDepth, RobotCar and TSS.
  arXiv  Detail & Related papers  (2021-04-07T17:58:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.