Sequential Planning via Anchored Robotic Keypoints

• URL: http://arxiv.org/abs/2606.30613v1
• Date: Mon, 29 Jun 2026 17:48:01 GMT
• Title: Sequential Planning via Anchored Robotic Keypoints
• Abstract summary: We present Sequential Planning via Anchored Robotic Keypoints, SPARK, a training-free neurosymbolic manipulation system.<n>A single Gemini call composes the plan as a typed behavior tree (BT) of composable kinematic primitives.<n>A second Gemini call proposes three alternative text prompts per object, SAM3 evaluates each, and we keep the prompt$to$label pair with the most confident detection.<n>The alternative prompts add +27.7 points on the spatial suite and +10.0 on the object suite, with the recovery loop adding +5.0 overall.
• Score: 3.176338968032152
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present Sequential Planning via Anchored Robotic Keypoints, SPARK, a training-free neurosymbolic manipulation system that reaches 43.7% on six LIBERO-PRO position \& task cells, more than doubling CaP-Agent0 and Vision-Language-Action (VLA) baselines. CaP-Agent0, a multi-turn code-generation agent, achieves 18.2% by re-querying an LLM at every turn, but its restart-from-scratch solution proves costly against minor policy failures. Perception is the layer that fails most under position and task changes so SPARK spends its computation there. A single Gemini call composes the plan as a typed behavior tree (BT) of composable primitives, each already containing the low-level control (motion, grasping, depth geometry) a code-generation agent would otherwise regenerate on every trial. The rest of the budget goes to perception: a second Gemini call proposes three alternative text prompts per object, SAM3 evaluates each, and we keep the prompt$\to$label pair with the most confident detection and a recovery loop then retries a failed primitive against freshly detected objects, with no new LLM call. The alternative prompts add +27.7 points on the spatial suite and +10.0 on the object suite, with the recovery loop adding +5.0 overall. SPARK runs the same primitives on three robot families (UR10e, Franka FR3, bimanual Franka) across nine unique tasks at twenty trials each, averaging 68%. Since the detector, planner, and controller modules sit behind the typed plan, they swap independently without training, and each primitive's checkable post-condition traces a failure to the corresponding module or a kinematic limit. Every trial logs a verified, labeled trajectory, so a training-free planner that already beats VLAs can supply the data those policies need without teleoperation. Project page: https://cwru-aism.github.io/spark-page/

Related papers

• A Schema Bounded Language Model for Refining Robot Policies Without Destabilizing Local Learning [0.0]
  Three robots share motion dynamics but use different LLM backends.<n>Each robot independently combines a large language model (LLM) policy agent, an Upper Confidence Bound (UCB) bandit, and a Double Deep Q-Network (Double DQN) controller.<n>The robots perform cross-LLM communication through a shared round summary containing policies, outcomes, and learning feedback.
  arXiv  Detail & Related papers  (2026-09-04T13:35:55Z)
• Codebook Agent: Amortized Topology Design for LLM Multi-Agent Systems [67.04448659688579]
  We develop a query-independent 16-entry codebook that ranks the top decoded candidates in a single batched forward pass.<n>With no iterative search and no message passing at test time, Codebook Agent is the most accurate method on all six benchmarks.
  arXiv  Detail & Related papers  (2026-09-02T08:10:22Z)
• Putting Registers to Work: Task Registers for Token Pruning in Vision Transformers [5.410304374735817]
  Token-pruning policies are usually designed for a single recognition pipeline, but pretrained Vision Transformers are reused across tasks with different spatial demands.<n>We ask which parts of a pruning policy transfer across image classification, semantic segmentation, and object detection.<n>For each pipeline, controlled probes freeze the no-pruning checkpoint and apply a series of parameter-free reduction criteria at one eligible layer at a time without retraining.<n>The probes reveal three differences: segmentation and detection rank the criteria differently, classification is especially sensitive to attention-based pruning in the earliest layers, and the dense tasks prefer opposite recovery endpoints.
  arXiv  Detail & Related papers  (2026-08-11T14:43:52Z)
• ValueFormer: A Causal Transformer Value Function with Stage-Aware Labels for Semi-Autonomous Vision-Language-Action Policies [0.576537956677604]
  We present ValueFormer, a compact policy-a causal transformer over a frozen DINOv3 backbone.<n> Failed episodes are labeled with a stage-aware, success-then-decay return that preserves the success curve before the failure stage, and detection is supervised from mistake intervals rather than a single failure time.
  arXiv  Detail & Related papers  (2026-08-03T23:46:39Z)
• Zero2Skill: Bootstrapping Robot Skills through Autonomous Data Collection, Training, and Deployment [53.913175773174636]
  Existing pipelines reduce human effort via self-resetting, VLM verification, or language-guided correction.<n>We present Zero2Skill, a human-robot symbiotic agentic system in which corrections are retained and reused across rounds.
  arXiv  Detail & Related papers  (2026-07-15T17:16:24Z)
• More Structure, Not More Capacity: Object-Centric Representations for Visuomotor Imitation Learning [54.65906330923846]
  We show that object-centric slot representations are a structured alternative to pre-trained vision models.<n>A dense patch grid with 16x the tokens performs no better than the global feature.<n>An explicit 2D spatial goal and native-resolution rendering raises the full system to 68.7$pm$4.2%, just below a privileged 3D-oracle upper bound.
  arXiv  Detail & Related papers  (2026-07-10T10:35:24Z)
• Doomed from the Start: Early Abort of LLM Agent Episodes via a Recall-Controlled Probe Cascade [14.63102636181275]
  Large language model (LLM) agents often waste inference compute by continuing multi-step trajectories that are already doomed to fail.<n>We study early failure prediction and inference-time early stopping for LLM agents using hidden-state probes.<n>We turn this signal into a recall-controlled abort cascade for reducing LLM agent inference costs.
  arXiv  Detail & Related papers  (2026-07-07T17:03:55Z)
• WatchAct: A Benchmark for Behavior-Grounded Robot Manipulation [49.31386176285509]
  WatchAct is a benchmark for robot manipulation grounded in observed human behavior.<n>WatchAct comprises 3,000 long-horizon instances across 14 tasks in four capability domains.<n>In both simulation and on a Franka Research 3 robot, current systems remain far from solving WatchAct.
  arXiv  Detail & Related papers  (2026-06-24T23:13:56Z)
• RHO: Your Coding Agent is Secretly a Roboticist [24.9798218503493]
  Code-as-Policies (CaP) has shown that large language models (LLMs) can write code to solve robotics tasks by composing perception, planning, and control primitives.<n>Recent CaP systems rely on multi-turn code-generation loops at test time, which is often infeasible for real-time robot control.<n>We introduce Robotics Harness Optimization (RHO), a novel paradigm in which tool-enabled coding agents, at training time, propose and search for interpretable, neurosymbolic multi-file policy repositories.
  arXiv  Detail & Related papers  (2026-06-15T09:30:12Z)
• AEGIS: A Backup Reflex for Physical AI [0.0]
  We introduce AEGIS, a selective escalation method that uses a lightweight probe on a weak policy's frozen activations to detect high-risk steps.<n>When the probe flags a step, control switches to a stronger separate policy, but only for the steps that need it.<n>On LIBERO-Spatial, AEGIS recovers 10.1% of the trajectories the weak policy alone loses, versus 4.6% for budget-matched blind escalation and 5.1% for a random-trigger placebo.
  arXiv  Detail & Related papers  (2026-06-04T19:09:22Z)
• From Attack Simulation to SIEM Rule: Deterministic Detection-as-Code Synthesis with Probe-Level Traceability [51.56484100374058]
  Security teams simulate attacks against their own systems to check whether their monitoring would catch a real intruder.<n>A human bridges that gap by hand, reading each finding and writing the corresponding Sigma rule.<n>We show this translation can be partially automated when probes are drawn from a locked corpus.
  arXiv  Detail & Related papers  (2026-06-03T14:26:25Z)
• Survive or Collapse: The Asymmetric Roles of Data Gating and Reward Grounding in Self-Play RL [76.45061154544568]
  Self-play reinforcement learning trains language models on their own generated tasks, co-evolving a proposer and solver without human labels.<n>Recent systems report strong reasoning gains, but collapse and instability are widely observed and poorly understood.<n>We argue instead that self-play stability is governed by two distinct levers: a data-level gate that decides which proposer-generated tasks enter the training pool, and the reward signal that updates the policy on tasks already admitted.
  arXiv  Detail & Related papers  (2026-05-21T09:19:23Z)
• TFGN: Task-Free, Replay-Free Continual Pre-Training Without Catastrophic Forgetting at LLM Scale [0.0]
  We introduce TFGN, an architectural overlay for transformer language models.<n> TFGN produces input-conditioned, parameter-efficient updates while leaving the rest of the transformer unchanged.<n>To our knowledge, TFGN is the first architecture that simultaneously closes catastrophic forgetting at LLM scale.
  arXiv  Detail & Related papers  (2026-05-14T16:46:26Z)
• PRISM: : Planning and Reasoning with Intent in Simulated Embodied Environments [59.07829883257003]
  Built on five multi-room apartments, PRISM structures 300 human-verified tasks into three capability tiers.<n>PRISM exposes an agent-agnostic executable action API that allows arbitrary agents to be evaluated end-to-end.
  arXiv  Detail & Related papers  (2026-05-12T04:59:47Z)
• VLAA-GUI: Knowing When to Stop, Recover, and Search, A Modular Framework for GUI Automation [98.38575149237442]
  VLAA-GUI is a modular GUI framework built around three integrated components.<n>A mandatory Completeness Verifier enforces UI-observable success criteria and verification at every finish step.<n>A mandatory Loop Breaker provides multi-tier switching interaction mode after repeated failures.
  arXiv  Detail & Related papers  (2026-04-23T07:42:37Z)
• Label-Free Cross-Task LoRA Merging with Null-Space Compression [50.63908869296697]
  We introduce Null-Space Compression (NSC) Merging, a label-free, output-agnostic method that sets merge weights from adapter geometry.<n>NSC achieves state-of-the-art performance across twenty heterogeneous vision tasks with balanced gains where prior methods overfit subsets of tasks.
  arXiv  Detail & Related papers  (2026-03-27T11:34:41Z)
• Altered Thoughts, Altered Actions: Probing Chain-of-Thought Vulnerabilities in VLA Robotic Manipulation [27.2709460237741]
  Recent Vision-Language-Action (VLA) models increasingly adopt chain-of-thought (CoT) reasoning, generating a natural-language plan before decoding motor commands.<n>We ask: which properties of this intermediate plan does the action decoder actually rely on, and can targeted corruption of the reasoning trace alone degrade a robot's physical task performance?<n>We design a taxonomy of seven text corruptions organized into three attacker tiers (blind noise, mechanical-semantic, and LLM-adaptive) and apply them to a state-of-the-art reasoning VLA across 40 tabletop manipulation tasks.
  arXiv  Detail & Related papers  (2026-03-13T07:02:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.