SUN: Persistent Programs For Language-Grounded Control-to-Learning-to-Real Policies

• URL: http://arxiv.org/abs/2608.31167v1
• Date: Mon, 31 Aug 2026 17:59:16 GMT
• Title: SUN: Persistent Programs For Language-Grounded Control-to-Learning-to-Real Policies
• Abstract summary: Existing protocols discard task semantics, leaving rewards hand-crafted and behavior drifting from what control verified.<n>We introduce Semantically UNified (SUN) Programs, typed executables where geometric and contact relations are defined once and compiled into aligned Model Predictive Control.<n>Our system, Kuafu, automatically synthesizes SUN Programs from language and scene semantics, screens feasibility via MPC, and retains semantics while training stage-conditioned policies.
• Score: 61.941546372624686
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Bridging model-based control and learned policies in long-horizon manipulation has harbored a silent disagreement: control executes specified objectives, learning amortizes that behavior into a reactive policy, yet existing protocols discard task semantics, leaving rewards hand-crafted and behavior drifting from what control verified.We introduce Semantically UNified (SUN) Programs, typed executables where geometric and contact relations are defined once and compiled into aligned Model Predictive Control (MPC) costs, satisfaction predicates, RL rewards, transition guards, and diagnostics. Our system, Kuafu, driven by large vision language systems, automatically synthesizes SUN Programs from language and scene semantics, screens feasibility via MPC, and retains semantics while training stage-conditioned policies. Across nine tasks, Kuafu achieves 82.03% macro-success, outperforming sparse-reward (35.67%) and Stage-BC (24.75%) baselines. At 8192-way scale, it generates 10.57x the successful trajectory time per hour of human teleoperation. With 500 trajectories per task, Kuafu data trains DP3 policies to 46.0% simulation success (vs. 22.4% for alternatives) and 34.7% on physical Franka and Kinova robots. These results establish that simulation-screened task semantics can effectively amortize control into robust policies, without demonstrations or manual dense rewards, unifying symbolic planning and data-driven execution.

Related papers

• Modality-Decoupled Federated Learning for Privacy-Preserving Embodied Intelligence in 6G [90.42669277637026]
  Sixth-generation (6G) wireless networks are expected to provide a key infrastructure for large-scale embodied intelligence.<n>This article proposes FedMVLA, a modality-decoupled FL framework for privacy-preserving embodied intelligence in 6G networks.
  arXiv  Detail & Related papers  (2026-09-09T01:36:44Z)
• Bridging Semantics and Physics with Constrained LLMs for Safe and Trustworthy Robotic Manipulation [6.975054201075641]
  A language-guided robot must do more than produce a plan that appears correct.<n>Plan may appear valid linguistically while being physically infeasible under kinematic and collision constraints.<n>We bridge this gap by formalizing the reasoning-execution boundary as a typed contract.
  arXiv  Detail & Related papers  (2026-08-29T17:26:20Z)
• Task-Conditioned Least-Privilege Learning for Executable Terminal and MCP Agents [45.88028371034407]
  Tool-using large language-model agents can complete a task while exercising authority that the user did not grant or the task does not need.<n>We propose a framework where each action is audited before execution and again from observed effects along six dimensions of risk.<n>We find that after training using this framework on Qwen3.5-4B over 1,500 tasks, the selected seed reaches 98.48% safe success across 2,896 evaluation episodes.
  arXiv  Detail & Related papers  (2026-08-18T22:07:23Z)
• Strategic Evaluation of Planning Strategies for LLM Agents in Cyber-Physical Systems [0.0]
  We introduce a benchmark built around planning-induced control trajectories.<n>It implements sequential, hierarchical, and search executors in a smart-grid demand-response system.<n>We show that forced search is the oracle in all five baseline seeds.
  arXiv  Detail & Related papers  (2026-08-04T22:52:21Z)
• PACT: Self-Evolving Physical Safety Alignment for Diffusion Policies in Embodied Manipulation [38.43792325467372]
  Existing approaches impose safety either prematurely during training or reactively via external guardrails at test time.<n>We propose Physical safety Alignment for Constrained Trajectories (PACT), a self-evolving post-training framework that projects pretrained diffusion policies onto constraint-feasible regions.<n>On simulated and real-world embodied manipulation benchmarks, PACT significantly reduces safety violations by 31.0% on average while improving task success by 30.7%.
  arXiv  Detail & Related papers  (2026-06-07T02:27:53Z)
• SignalClaw: LLM-Guided Evolutionary Synthesis of Interpretable Traffic Signal Control Skills [8.299375016847524]
  Traffic signal control TSC requires strategies that are both effective and interpretable for deployment.<n>We present SIGNALCLAW, a framework that uses large language models as evolutionary skill generators.<n>Each skill includes rationale, selection guidance, and executable code, making policies human-inspectable and self-documenting.
  arXiv  Detail & Related papers  (2026-04-07T07:35:45Z)
• Verified Critical Step Optimization for LLM Agents [67.05296684575445]
  Critical Step Optimization focuses preference learning on verified critical steps.<n>Method starts from failed policy trajectories rather than expert demonstrations.<n>Experiments on GAIA-Text-103 and XBench-DeepSearch show that CSO achieves 37% and 26% relative improvement over the SFT baseline.
  arXiv  Detail & Related papers  (2026-02-03T11:41:02Z)
• Learning Policy Representations for Steerable Behavior Synthesis [80.4542176039074]
  Given a Markov decision process (MDP), we seek to learn representations for a range of policies to facilitate behavior steering at test time.<n>We show that these representations can be approximated uniformly for a range of policies using a set-based architecture.<n>We use variational generative approach to induce a smooth latent space, and further shape it with contrastive learning so that latent distances align with differences in value functions.
  arXiv  Detail & Related papers  (2026-01-29T21:52:06Z)
• Opening the Sim-to-Real Door for Humanoid Pixel-to-Action Policy Transfer [59.02729900344616]
  GPU-accelerated, photorealistic simulation has opened a scalable data-generation path for robot learning.<n>We develop a teacher-student-bootstrap learning framework for vision-based humanoid loco-manipulation.<n>This represents the first humanoid sim-to-real policy capable of diverse articulated loco-manipulation using pure RGB perception.
  arXiv  Detail & Related papers  (2025-11-30T20:07:13Z)
• Zero-Shot Whole-Body Humanoid Control via Behavioral Foundation Models [71.34520793462069]
  Unsupervised reinforcement learning (RL) aims at pre-training agents that can solve a wide range of downstream tasks in complex environments.<n>We introduce a novel algorithm regularizing unsupervised RL towards imitating trajectories from unlabeled behavior datasets.<n>We demonstrate the effectiveness of this new approach in a challenging humanoid control problem.
  arXiv  Detail & Related papers  (2025-04-15T10:41:11Z)
• Local Policies Enable Zero-shot Long-horizon Manipulation [80.1161776000682]
  We introduce ManipGen, which leverages a new class of policies for sim2real transfer: local policies.<n>ManipGen outperforms SOTA approaches such as SayCan, OpenVLA, LLMTrajGen and VoxPoser across 50 real-world manipulation tasks by 36%, 76%, 62% and 60% respectively.
  arXiv  Detail & Related papers  (2024-10-29T17:59:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.