REPOT: Recoverable Program-of-Thought via Checkpoint Repair

• URL: http://arxiv.org/abs/2605.30052v1
• Date: Thu, 28 May 2026 15:03:17 GMT
• Title: REPOT: Recoverable Program-of-Thought via Checkpoint Repair
• Abstract summary: One-shot Program-of-Thought (PoT) emits a Python program that prints a primitive-action plan.<n>We introduce RePoT: a deterministic verified replay that walks the plan through the environment to its first invalid transition.<n>RePoT beats PoT by +3 to +11pp across four closed-model configurations.
• Score: 0.03038255798961237
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: One-shot Program-of-Thought (PoT) emits a Python program that prints a primitive-action plan; a single invalid action silently invalidates the trajectory. We introduce RePoT (Recoverable PoT): a deterministic verified replay that walks the plan through the environment to its first invalid transition, then one LLM call that resumes from the verified prefix. RePoT costs at most one extra LLM call on the ~14% of problems where PoT fails. RePoT beats PoT by +3 to +11pp across four closed-model configurations on PuzzleZoo-775 and peaks at 96.9% vs 86.3% on gpt-5.4-mini-medium; against the matched-budget PoT-retry baseline, RePoT wins decisively on Gemini (+3.8pp, 95% CI [+2.2,+5.4]), is within sampling noise on GPT-medium and Claude, and loses on GPT-mini -- a capability-scaling pattern we begin to address with Adaptive RePoT, a rule-based dispatcher that routes between suffix repair and a fresh PoT retry based on verified-prefix length (preliminary). We replicate on PlanBench Blocksworld (+1.1 to +11.4pp) and on four open-weights models (+3.3 to +20.0pp on three of four). On Derail-550, our controlled recovery benchmark, every condition with access to checkpoint information clears >=30% on GPT-medium and >=70% on Gemini, vs <=3.1% for error-only feedback -- showing that checkpoint information, not the specific verified-prefix tail, is the load-bearing recovery signal.

Related papers

• ReDraft, Don't Just Distill: Reference-Driven Revision for Continual VLLM Post-Training [66.72708893922605]
  Post-training of large multimodal models should add new capabilities while preserving those from pre-training.<n>SFT gives explicit target supervision that learns a task from near-zero accuracy, but its off-policy targets move the model far enough to cause forgetting.<n>We introduce ReDraft, which obtains both from the model's own failures.
  arXiv  Detail & Related papers  (2026-09-15T04:59:03Z)
• On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability [57.03393882933515]
  We describe the architecture and ablations of Qwen3.8-Flash-Next, a sparse mixture-of-experts model with 125B parameters, 6B activated per token, and 51B parameters of n-gram embedding tables held off the accelerator.<n>On fourteen pre-training benchmarks the model leads the 397B-A17B predecessor on eight and trails it on the rest by at most 2.6 points, at 1/3 the activated parameters, 1/3 the training tokens, and roughly 1/9 the training FLOPs.
  arXiv  Detail & Related papers  (2026-08-31T06:35:07Z)
• Sequential Planning via Anchored Robotic Keypoints [3.176338968032152]
  We present Sequential Planning via Anchored Robotic Keypoints, SPARK, a training-free neurosymbolic manipulation system.<n>A single Gemini call composes the plan as a typed behavior tree (BT) of composable kinematic primitives.<n>A second Gemini call proposes three alternative text prompts per object, SAM3 evaluates each, and we keep the prompt$to$label pair with the most confident detection.<n>The alternative prompts add +27.7 points on the spatial suite and +10.0 on the object suite, with the recovery loop adding +5.0 overall.
  arXiv  Detail & Related papers  (2026-06-29T17:48:01Z)
• CRMA: A Spectrally-Bounded Backbone for Modular Continual Fine-Tuning of LLMs [0.0]
  We introduce CRMA, a residual adapter whose internal mixing matrix M is doubly-stochastic at every forward pass via Sinkhorn normalization.<n>On Mistral-7B across 5 domains 3 seeds, modular per-task LoRA mediates a backbone loss-relative drift from +42.96% +/- 5.5 (naive fine-tuning) to -0.17% +/- 0.17, with disjoint per-seed ranges, and improves prior-task holdout loss by 1.99% +/- 0.54 over a matched frozen-substrate baseline.
  arXiv  Detail & Related papers  (2026-05-29T21:50:23Z)
• AgentModernize: Preserving Business Logic in Legacy Modernization with Multi-Agent LLMs and Behavioral Specification Graphs [0.0]
  We present AgentModernize, a framework that treats modernization as a behavioral preservation problem.<n>Four specialized agents handle extraction, specification, code generation, and validation.<n> Behavioral Specification Graph captures 91.2% of gold-standard rules, confirming that the bottleneck is code generation, not extraction.
  arXiv  Detail & Related papers  (2026-05-17T16:39:48Z)
• Perturbation Dose Responses in Recursive LLM Loops: Raw Switching, Stochastic Floors, and Persistent Escape under Append, Replace, and Dialog Updates [0.0]
  We study how much injected text is needed to move a settled loop somewhere else, and whether that move lasts.<n>Under a 12,000-character tail clip, destination-coherent persistence plateaus near 16 percent and retained source-basin escape near 36 percent at dose 400.<n>A four-step falsification battery recasts the high-dose destination-coherent dip as a finite-horizon, endpoint-definition-sensitive feature.
  arXiv  Detail & Related papers  (2026-05-04T05:16:43Z)
• Universal Transformers Need Memory: Depth-State Trade-offs in Adaptive Recursive Reasoning [0.0]
  We study learned memory tokens as computational scratchpad for a single-block Universal Transformer (UT) on SudokuExtreme.<n>We find that memory tokens are empirically necessary: across all configurations tested, no configuration without memory tokens non-trivial performance.
  arXiv  Detail & Related papers  (2026-04-23T18:30:01Z)
• Reasoning Topology Matters: Network-of-Thought for Complex Reasoning Tasks [5.523132953818281]
  Chain-of-Thought (CoT) produces linear traces, while Tree-of-Thought (ToT) performs branching search.<n>We propose Network-of-Thought (NoT), a framework that models reasoning as a directed graph with typed nodes and edges.
  arXiv  Detail & Related papers  (2026-03-21T09:32:28Z)
• Squeeze the Soaked Sponge: Efficient Off-policy Reinforcement Finetuning for Large Language Model [56.92219181993453]
  We propose Reincarnating Mix-policy Proximal Policy Gradient (ReMix) to enable on-policyRFT methods like PPO and GRPO to leverage off-policy data.<n>ReMix consists of three major components: (1) Mix-policy proximal policy gradient with an increased Update-To-Data (UTD) ratio for efficient training; (2) KL-Convex policy constraint to balance the trade-off between stability and flexibility; (3) Policy reincarnation to achieve a seamless transition from efficient early-stage learning to steady improvement.
  arXiv  Detail & Related papers  (2025-07-09T14:29:45Z)
• InstructRetro: Instruction Tuning post Retrieval-Augmented Pretraining [47.60376031955207]
  Retro 48B is the largest large language model pretrained with retrieval. InstructRetro demonstrates significant improvement over the instruction tuned GPT on a wide range of zero-shot tasks.
  arXiv  Detail & Related papers  (2023-10-11T17:59:05Z)
• The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuning [50.75534397373867]
  Language models (LMs) with less than 100B parameters are known to perform poorly on chain-of-thought (CoT) reasoning. In this work, we aim to equip smaller LMs with the step-by-step reasoning capability by instruction tuning with CoT rationales.
  arXiv  Detail & Related papers  (2023-05-23T13:14:59Z)
• Retrosynthetic Planning with Dual Value Networks [107.97218669277913]
  We propose a novel online training algorithm, called Planning with Dual Value Networks (PDVN) PDVN alternates between the planning phase and updating phase to predict the synthesizability and cost of molecules. On the widely-used USPTO dataset, our PDVN algorithm improves the search success rate of existing multi-step planners.
  arXiv  Detail & Related papers  (2023-01-31T16:43:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.