World Models as Group Actions

• URL: http://arxiv.org/abs/2605.24578v1
• Date: Sat, 23 May 2026 13:42:35 GMT
• Title: World Models as Group Actions
• Abstract summary: We argue that action faithfulness should be understood through the compositional structure of actions.<n>We formalize action-conditioned world modeling as realizing a group action on the state space.<n>Our method consistently improves both Group-Action Consistency (GAC) and Group-Action Robustness (GAR) in state-of-the-art video world models.
• Score: 70.0506723637968
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Video world models have achieved strong visual realism, but this does not ensure that their dynamics are truly governed by actions. In this work, we argue that action faithfulness should be understood through the compositional structure of actions, which in many embodied settings follows a group structure (e.g., SE(2) for navigation). Based on this insight, we formalize action-conditioned world modeling as realizing a group action on the state space, providing a principled criterion for evaluating dynamics beyond visual quality. To operationalize this framework, we propose a unified approach that enforces identity, inverse, and composition consistency via latent-space regularization with synthesized supervision, avoiding additional data collection. We further introduce two metrics: Group-Action Consistency (GAC) and Group-Action Robustness (GAR), to evaluate structural correctness and rollout stability. Extensive experimental results show that our method consistently improves both GAC and GAR in state-of-the-art video world models without degrading perceptual quality.

Related papers

• Scaling Behavior Foundation Model for Humanoid Robots [77.31115920959364]
  Behavior Foundation Models (BFMs) have emerged as a promising solution to address challenges by leveraging large-scale behavioral data.<n>We show that our approach yields significant improvements in control fidelity and task generalization.<n>These results establish BFM as a principled and effective foundation for scalable and general-purpose humanoid control.
  arXiv  Detail & Related papers  (2026-07-16T16:08:27Z)
• Co-VLA: Coordination-Aware Structured Action Modeling for Dual-Arm Vision-Language-Action Systems [30.96972509327388]
  Co-VLA is a coordination-aware bimanual manipulation framework.<n>We introduce explicit structure at the action generation level with a modular coordination-aware loss.<n>Experiments show Co-VLA significantly outperforms monolithic baselines.
  arXiv  Detail & Related papers  (2026-06-18T14:28:37Z)
• World Action Models: The Next Frontier in Embodied AI [123.5787299299832]
  Vision-Language-Action (VLA) models have achieved strong semantic generalization for embodied policy learning.<n>They learn reactive observation-to-action mappings without explicitly modeling how the physical world evolves under intervention.<n>A growing body of work addresses this limitation by integrating world models, predictive models of environment dynamics, into the action generation pipeline.
  arXiv  Detail & Related papers  (2026-05-12T13:10:52Z)
• HarmoWAM: Harmonizing Generalizable and Precise Manipulation via Adaptive World Action Models [58.191567345416836]
  World Action Models (WAMs) have emerged as a promising paradigm for robot control by modeling physical dynamics.<n>HarmoWAM fully leverages a world model to unify predictive and reactive control, enabling both general transit and precise manipulation.<n>We construct three training-unseen test environments across six real-world robotic tasks, covering variations in background, position, and object semantics.
  arXiv  Detail & Related papers  (2026-05-11T17:59:56Z)
• Beyond State Consistency: Behavior Consistency in Text-Based World Models [58.216587360435305]
  We introduce a new behavior-aligned training paradigm aimed at improving the functional consistency between the world model and the real environment.<n> Experiments on WebShop and TextWorld show that BehR-based training improves long-term alignment in several settings.
  arXiv  Detail & Related papers  (2026-04-15T12:56:45Z)
• Beyond the Academic Monoculture: A Unified Framework and Industrial Perspective for Attributed Graph Clustering [75.50670592447219]
  Attributed Graph Clustering (AGC) is a fundamental unsupervised task that partitions nodes into cohesive groups by jointly modeling structural topology and node attributes.<n>This survey provides a comprehensive, industrially grounded review of AGC from three complementary perspectives.
  arXiv  Detail & Related papers  (2026-03-21T14:15:34Z)
• Exploring the Temporal Consistency for Point-Level Weakly-Supervised Temporal Action Localization [66.80402022104074]
  Point-supervised Temporal Action Localization (PTAL) adopts a lightly frame-annotated paradigm (textiti.e., labeling only a single frame per action instance) to train a model to locate action instances within unsupervised videos.<n>Most existing approaches design the task head of models with only a point-trimmed snippet-level classification, without explicit modeling of understanding temporal relationships among frames of an action.<n>We propose a multi-task learning framework that fully utilizes point supervision to boost the model's temporal understanding capability for action localization.
  arXiv  Detail & Related papers  (2026-02-05T14:46:21Z)
• DRFormer: A Dual-Regularized Bidirectional Transformer for Person Re-identification [37.61695934257133]
  Fine-grained discriminative details and global semantic features can contribute to solving person re-identification challenges.<n>Vision foundation models excel at mining local textures, and vision-language models capture strong global semantic difference.<n>We propose a framework to synergize their strengths by a textbfDual-textbfRegularized Bidirectional textbfTransformer.
  arXiv  Detail & Related papers  (2026-02-01T06:59:53Z)
• Object-Centric World Models for Causality-Aware Reinforcement Learning [13.063093054280946]
  We propose emph Transformer Imagination with CAusality-aware reinforcement learning (ASTICA)<n>A unified framework in which object-centric Transformers serve as the world model and causality-aware policy and value networks.<n>Experiments on object-rich benchmarks demonstrate that STICA consistently outperforms state-of-the-art agents in both sample efficiency and final performance.
  arXiv  Detail & Related papers  (2025-11-18T08:53:09Z)
• Chain-of-Action: Trajectory Autoregressive Modeling for Robotic Manipulation [37.748111048944274]
  Chain-of-Action (CoA) is a visuo-motor policy paradigm built upon Trajectory Autoregressive Modeling.<n>CoA generates an entire trajectory by explicit backward reasoning with task-specific goals.<n>We observe CoA the state-of-the-art performance across 60 RLBench tasks and 8 real-world manipulation tasks.
  arXiv  Detail & Related papers  (2025-06-11T17:59:13Z)
• From Local Cues to Global Percepts: Emergent Gestalt Organization in Self-Supervised Vision Models [7.7536110932446265]
  We investigate whether modern vision models show similar behaviors, and under what training conditions these emerge.<n>We find that Vision Transformers (ViTs) trained with Masked Autoencoding (MAE) exhibit activation patterns consistent with Gestalt laws.<n>We introduce the Distorted Spatial Relationship Testbench (DiSRT), which evaluates sensitivity to global spatial perturbations while preserving local textures.
  arXiv  Detail & Related papers  (2025-05-31T21:35:54Z)
• Unifying Perspectives: Plausible Counterfactual Explanations on Global, Group-wise, and Local Levels [2.494108084558292]
  We propose a gradient-based optimization method for differentiable models that generates Counterfactual Explanations in a unified manner.<n>We especially enhance GWCF generation by combining instance grouping and counterfactual generation into a single efficient process.<n>Our results demonstrate the method's effectiveness in balancing validity, proximity, and plausibility while optimizing group granularity.
  arXiv  Detail & Related papers  (2024-05-27T20:32:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.