World-Task Factorization for Robot Learning

• URL: http://arxiv.org/abs/2606.02027v1
• Date: Mon, 01 Jun 2026 10:16:07 GMT
• Title: World-Task Factorization for Robot Learning
• Abstract summary: We study what we argue is the most fundamental factorization in robotics: separating the world from the task.<n>World factors are properties of the embodied system and the environment.<n>Task factors are defined by the task's logic over what the world admits.
• Score: 14.616160772427847
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Robot learning must produce policies that generalize to new combinations of constraints, teammates, and environments. To achieve this, we must structurally factor the policy, which is a choice that dictates what generalizes, what requires retraining, and what remains entangled. Existing methods span a wide spectrum, from expecting structure to emerge from data scaling, to hand-designing it via hierarchies, skill libraries or learned specializations. In this paper, we study what we argue is the most fundamental factorization in robotics: separating the world from the task. We investigate the conditions under which this factorization is principled. World factors are properties of the embodied system and the environment; they exist independently of intent. Task factors are defined by the task's logic over what the world admits. We formalize this asymmetry through Bayesian model evidence: it aligns with the data-generating process, maintains high likelihood through an analytical world model, and reduces the Occam razor's penalty on task parameters. We instantiate this factorization by pairing AICON, a differentiable graph of recursive estimators and interconnections that is compositional, operates without task-specific data, and propagates cost gradients to actuators, with a compact, learned policy that modulates gradient paths. Gradients serve as the interface between the two factors: they carry world structure through the graph and task structure through costs, enabling low-dimensional learning while preserving structural generalization. We test the world/task factorization across three problems that encompass heterogeneous robots, environments, task logic and sensorimotor modalities. Our framework outperforms end-to-end baselines and analytical heuristics in all settings, generalizes zero-shot to out-of-distribution configurations, and transfers to real hardware without retraining.

Related papers

• What Makes Good Agentic Data? An ACE Lens on Data Generation for LLM Agents [81.23160457293714]
  LLM agents increasingly rely on generated interaction data to learn how to interact with external environments.<n>Existing work spans many agent domains, but domain-centered organization and heterogeneous evaluation often obscure common generation mechanisms.<n>This work develops a two-level framework for the field.
  arXiv  Detail & Related papers  (2026-08-27T15:43:50Z)
• Evolutionary Task Discovery: Advancing Reasoning Frontiers via Skill Composition and Complexity Scaling [10.652489725330401]
  Evoutionary Task Discovery (EvoTD) is a framework that treats data as a directed search over a dual-axis manifold of Algorithmic Skills and Complexity Attributes.<n>We introduce structured evolutionary operators to navigate this space: a Crossover operator that synthesizes novel skill compositions to enhance diversity, and a Parametric Mutation operator that scales structural constraints.<n>EvoTD delivers substantial reasoning gains that generalize consistently across model architectures, pretraining regimes, and scales.
  arXiv  Detail & Related papers  (2026-05-12T07:26:40Z)
• Towards Generalizable Robotic Data Flywheel: High-Dimensional Factorization and Composition [15.755384541529098]
  The lack of sufficiently diverse data is a major bottleneck for generalist robotic models.<n>We propose F-ACIL, a factor-aware iterative learning framework that enables structured data factorization.<n>With extensive real-world experiments, we show that F-ACIL can achieve more than 45% performance gains.
  arXiv  Detail & Related papers  (2026-03-26T16:00:39Z)
• Iterative Compositional Data Generation for Robot Control [8.888011063034517]
  We propose a semantic compositional diffusion transformer that factorizes transitions into robot-, object-, obstacle-, and objective-specific components.<n>We show that our model can zero-shot generate high-quality transitions from which we can learn control policies for unseen task combinations.
  arXiv  Detail & Related papers  (2025-12-11T18:20:49Z)
• ExoPredicator: Learning Abstract Models of Dynamic Worlds for Robot Planning [77.49815848173613]
  We propose a framework for abstract world models that jointly learns symbolic state representations and causal processes for both endogenous actions and mechanisms.<n>Across five simulated tabletop robotics environments, the learned models enable fast planning that generalizes to held-out tasks with more objects and more complex goals, outperforming a range of baselines.
  arXiv  Detail & Related papers  (2025-09-30T13:44:34Z)
• Learning Causal Structure Distributions for Robust Planning [53.753366558072806]
  We find that learning the functional relationships while accounting for the uncertainty about the structural information leads to more robust dynamics models.<n>This in contrast with common model-learning methods that ignore the causal structure and fail to leverage the sparsity of interactions in robotic systems.<n>We show that our model can be used to learn the dynamics of a robot, which together with a sampling-based planner can be used to perform new tasks in novel environments.
  arXiv  Detail & Related papers  (2025-08-08T22:43:17Z)
• Shortcut Learning in Generalist Robot Policies: The Role of Dataset Diversity and Fragmentation [117.54237701533805]
  Generalist robot policies trained on large-scale datasets such as Open X-Embodiment (OXE) demonstrate strong performance across a wide range of tasks.<n>They often struggle to generalize beyond the distribution of their training data.<n>We identify shortcut learning as a key impediment to generalization.
  arXiv  Detail & Related papers  (2025-08-08T16:14:01Z)
• The Coverage Principle: A Framework for Understanding Compositional Generalization [31.762330857169914]
  We show that models relying primarily on pattern matching for compositional tasks cannot reliably generalize beyond substituting fragments that yield identical results when used in the same contexts.<n>We demonstrate that this framework has a strong predictive power for the generalization capabilities of Transformers.
  arXiv  Detail & Related papers  (2025-05-26T17:55:15Z)
• Policy Architectures for Compositional Generalization in Control [71.61675703776628]
  We introduce a framework for modeling entity-based compositional structure in tasks. Our policies are flexible and can be trained end-to-end without requiring any action primitives.
  arXiv  Detail & Related papers  (2022-03-10T06:44:24Z)
• CausalWorld: A Robotic Manipulation Benchmark for Causal Structure and Transfer Learning [138.40338621974954]
  CausalWorld is a benchmark for causal structure and transfer learning in a robotic manipulation environment. Tasks consist of constructing 3D shapes from a given set of blocks - inspired by how children learn to build complex structures.
  arXiv  Detail & Related papers  (2020-10-08T23:01:13Z)
• MetaPerturb: Transferable Regularizer for Heterogeneous Tasks and Architectures [61.73533544385352]
  We propose a transferable perturbation, MetaPerturb, which is meta-learned to improve generalization performance on unseen data. As MetaPerturb is a set-function trained over diverse distributions across layers and tasks, it can generalize heterogeneous tasks and architectures.
  arXiv  Detail & Related papers  (2020-06-13T02:54:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.