Inference-Time Robot Behavior Steering through Physically-Aware Reconfiguration of Task-Structure

• URL: http://arxiv.org/abs/2606.26588v1
• Date: Thu, 25 Jun 2026 04:20:08 GMT
• Title: Inference-Time Robot Behavior Steering through Physically-Aware Reconfiguration of Task-Structure
• Abstract summary: ReStruct steers a wide range of preferences, from object-centric specifications to temporal-logic constraints.<n>It exceeds VLA models in both task success and preference-following by up to 25%.
• Score: 13.442250004542581
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A central challenge in deploying learned robot policies is inference-time behavior steering: redirecting a policy at test time to satisfy user preferences not anticipated during training, without retraining. Existing methods fail in two modes: end-to-end methods require fine-tuning or expert-level guidance, while neuro-symbolic methods rely on predefined symbols whose edits can result in logically reasonable but physically infeasible plans. To address this challenge, we propose ReStruct, which builds upon a neural automaton policy that decomposes a visuomotor policy into a high-level state-machine skeleton capturing task structure and a low-level continuous controller represented as a residual policy. Specifically, ReStruct adopts the automaton to represent the preference and incorporates it into the skeleton through a synchronous product, thereby reconfiguring the task structure. With the controller kept frozen, the action priors provided by the skeleton are updated accordingly to enable physically-aware control under a modified task structure. Extensive experiments from simulation and real-world show that ReStruct steers a wide range of preferences, from object-centric specifications to temporal-logic constraints, and after steering surpasses existing methods, exceeding VLA models in both task success and preference-following by up to 25%.

Related papers

• CSymPlan: Certified Symbolic Planning and Control for High-DOF Manipulators [4.257148651564725]
  CSymPlan is a symbolic planning and control framework for high-DOF manipulators.<n> offline implementation precomputes certified reach-avoid feedback policies for known workspaces.<n> online implementation synthesizes or updates symbolic policies at runtime from changing task and perception information.<n>Both implementations complete reach-avoid tasks with zero safety violations.
  arXiv  Detail & Related papers  (2026-08-24T08:45:37Z)
• CoStream: Composing Simple Behaviors for Generalizable Complex Manipulation [61.25264898597319]
  We show that complex manipulation capabilities can emerge naturally from the composition of simple, independent behaviors.<n>We propose ourshort, a framework orchestrating foundation models and diverse sensing modalities into multiple composable core behaviors.<n>We demonstrate ourshort on 8 real-world tasks spanning everyday manipulation and precision assembly, with the strongest gains in contact-rich assembly and object transfer.
  arXiv  Detail & Related papers  (2026-06-24T22:25:19Z)
• Mind Your Steps: A General Learning Framework for Accurate Humanoid Foothold Tracking [78.30338456206984]
  We introduce a novel framework for training general-purpose 3D foothold-tracking policies.<n>By dynamically providing footstep support through a goal sampler, this method enables the learned policy to be agnostic to specific terrains.<n>Our new target representation effectively mitigates challenges arising in the real world, such as noisy and inaccurate pose estimation and foot contact estimation.
  arXiv  Detail & Related papers  (2026-06-06T16:44:56Z)
• Learning Reactive Dexterous Grasping via Hierarchical Task-Space RL Planning and Joint-Space QP Control [50.28263951510334]
  We propose a hybrid hierarchical control framework for reactive dexterous grasping.<n>The proposed framework explicitly decouples high-level spatial intent from low-level joint execution.<n>We extensively validate the proposed framework through a rigorous simulation-to-reality pipeline.
  arXiv  Detail & Related papers  (2026-05-05T04:49:38Z)
• Emergent Neural Automaton Policies: Learning Symbolic Structure from Visuomotor Trajectories [14.475289544849007]
  ENAP (Emergent Neural Automaton Policy) is a framework that allows a bi-level neuro-symbolic policy adaptively emerge from visuomotor demonstrations.<n>By explicitly modeling the task structure with discrete transitions and continuous residuals, ENAP achieves high sample efficiency and interpretability without requiring task-specific labels.
  arXiv  Detail & Related papers  (2026-03-26T20:50:36Z)
• Closed-Loop Verbal Reinforcement Learning for Task-Level Robotic Planning [1.6641413441634143]
  We propose a new Verbal Reinforcement Learning framework for interpretable task-level planning in mobile robotic systems.<n>The framework follows a closed-loop architecture that enables iterative policy improvement through interaction with the physical environment.<n>We validate the proposed framework on a real mobile robot performing a multi-stage manipulation and navigation task under execution uncertainty.
  arXiv  Detail & Related papers  (2026-03-23T16:28:36Z)
• LDHP: Library-Driven Hierarchical Planning for Non-prehensile Dexterous Manipulation [3.829270241398111]
  Non-prehensile manipulation is essential for handling thin, large, or otherwise ungraspable objects in unstructured settings.<n>Prior planning and search-based methods often rely on ad-hoc manual designs or generate physically unrealizable motions.<n>We propose a library-driven hierarchical planner (LDHP) that makes executability a first-class design goal.
  arXiv  Detail & Related papers  (2026-03-14T08:54:49Z)
• RADAR: Closed-Loop Robotic Data Generation via Semantic Planning and Autonomous Causal Environment Reset [48.645870795753105]
  We introduce Robust Autonomous Data Acquisition for Robotics (RADAR)<n>RADAR is a fully autonomous, closed-loop data generation engine that completely removes human intervention from the collection cycle.<n>In simulation, our framework achieves up to 90% success rates on complex, long-horizon tasks.
  arXiv  Detail & Related papers  (2026-03-12T11:18:52Z)
• Zero-Shot Instruction Following in RL via Structured LTL Representations [50.41415009303967]
  We study instruction following in multi-task reinforcement learning, where an agent must zero-shot execute novel tasks not seen during training.<n>In this setting, linear temporal logic has recently been adopted as a powerful framework for specifying structured, temporally extended tasks.<n>While existing approaches successfully train generalist policies, they often struggle to effectively capture the rich logical and temporal structure inherent in specifications.
  arXiv  Detail & Related papers  (2026-02-15T23:22:50Z)
• ToolSelf: Unifying Task Execution and Self-Reconfiguration via Tool-Driven Intrinsic Adaptation [60.25542764389203]
  Agentic systems powered by Large Language Models (LLMs) have demonstrated remarkable potential in tackling complex, long-horizon tasks.<n>Existing approaches, relying on manual orchestration or runtime-based patches, often struggle with poor generalization and fragmented optimization.<n>We propose ToolSelf, a novel paradigm enabling tool-driven self-readjustment.
  arXiv  Detail & Related papers  (2026-02-08T09:27:18Z)
• Constrained Decoding for Robotics Foundation Models [12.916330118607918]
  We introduce SafeDec, a constrained decoding framework for autoregressive robot foundation models.<n>Task-specific safety rules are expressed as Signal Temporal Logic (STL) formulas and are enforced at inference time with minimal overhead.<n>Our method ensures that generated actions provably satisfy STL specifications under assumed dynamics at runtime without retraining.
  arXiv  Detail & Related papers  (2025-09-01T19:17:40Z)
• Learning Robust Policy against Disturbance in Transition Dynamics via State-Conservative Policy Optimization [63.75188254377202]
  Deep reinforcement learning algorithms can perform poorly in real-world tasks due to discrepancy between source and target environments. We propose a novel model-free actor-critic algorithm to learn robust policies without modeling the disturbance in advance. Experiments in several robot control tasks demonstrate that SCPO learns robust policies against the disturbance in transition dynamics.
  arXiv  Detail & Related papers  (2021-12-20T13:13:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.