STRIPS-WM: Learning Grounded Propositional STRIPS-style World Models from Images

• URL: http://arxiv.org/abs/2606.06832v1
• Date: Fri, 05 Jun 2026 02:16:50 GMT
• Title: STRIPS-WM: Learning Grounded Propositional STRIPS-style World Models from Images
• Abstract summary: STRIPS-WM is a framework for learning STRIPS-style world models directly from visual transitions.<n>It learns latent binary predicates and one grounded propositional operator per action label.<n>It forms a symbolic action model with sparse preconditions and add/delete effects.
• Score: 1.3535770763481905
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Robots performing long-horizon visual manipulation observe high-dimensional images, but successful plans depend on action-relevant facts: what can be done now and what changes afterward. A useful planning representation should discard irrelevant visual details while preserving action applicability and effects. Classical task planners exploit this structure through symbolic operators with preconditions and effects, but obtaining such representations from raw visual experience remains challenging. We study a visual task-planning setting in which a robot receives only image transitions: the current image, executed high-level action, and the resulting image. At test time, given a start image and a goal image, the robot must produce a sequence of high-level actions that reaches the goal. To address this problem, we introduce STRIPS-WM, a framework for learning image-grounded STRIPS-style world models directly from visual transitions. STRIPS-WM first induces a finite abstract transition graph from images, then learns latent binary predicates and one grounded propositional operator per action label. The learned operators form a symbolic action model with sparse preconditions and add/delete effects. Finally, the learned predicates are distilled into a visual encoder, enabling classical planning directly from novel start and goal images. Experiments on visual rearrangement tasks show that STRIPS-WM improves image-to-plan success over the tested visual rollout, latent graph-search and latent-symbolic baselines.

Related papers

• Breaking the Vision-Action Shortcut: Latent Interface Training for Generalizable Robotics Foundation Models [3.215979391291133]
  We propose a framework-agnostic two-stage strategy for training robot foundation models.<n> Stage 1 establishes a spatial-goal-conditioned action prior to images, then constrains visual conditioning through a pose-supervised latent interface.<n>Stage 2 introduces a latent interface that aggregates visual and semantic representations and serves as the pretrained action expert's only visual conditioning pathway.
  arXiv  Detail & Related papers  (2026-09-11T09:42:44Z)
• RepWAM: World Action Modeling with Representation Visual-Action Tokenizers [120.87642891397853]
  RepWAM is a representation-centric world action model built on representation visual-action tokenizers.<n>We train a representation visual-action tokenizer that maps visual inputs into aligned visual and latent action tokens.<n>We then pretrain our WAM to jointly model future visual states and the latent actions that connect them under language instructions.
  arXiv  Detail & Related papers  (2026-06-11T17:59:43Z)
• SWEET: Sparse World Modeling with Image Editing for Embodied Task Execution [61.612676324369595]
  We study whether image editing models can serve as sparse visual world models for robot manipulation.<n>We propose SWEET, a one-shot prediction visual planning framework.<n>Experiments on DROID and RoboMimic show that SWEET improves prediction across seen and unseen scenes.
  arXiv  Detail & Related papers  (2026-05-19T03:54:46Z)
• A Saccade-inspired Approach to Image Classification using Vision Transformer Attention Maps [0.9332987715848716]
  We draw inspiration from the human visual system to create smarter image processing models.<n>Using DINO, a self-supervised Vision Transformer, we explore a saccade inspired method to focus the processing of information on key regions in visual space.<n>This selective-processing strategy preserves most of the full-image classification performance and can even outperform it in certain cases.
  arXiv  Detail & Related papers  (2026-03-10T12:54:55Z)
• SAGE: Scene Graph-Aware Guidance and Execution for Long-Horizon Manipulation Tasks [3.688836621357062]
  Long-horizon manipulation tasks involve extended action sequences and complex object interactions.<n>We propose SAGE, a novel framework for Scene Graph-Aware Guidance and Execution in Long-Horizon Manipulation Tasks.<n> SAGE consists of two key components: (1) a scene graph-based task planner that uses VLMs and LLMs to parse the environment and reason about physically-grounded scene state transition sequences, and (2) a decoupled structural image editing pipeline that controllably converts each target sub-goal graph into a corresponding image.
  arXiv  Detail & Related papers  (2025-09-26T06:14:55Z)
• ViCrit: A Verifiable Reinforcement Learning Proxy Task for Visual Perception in VLMs [98.27348724529257]
  We introduce ViCrit (Visual Caption Hallucination Critic), an RL proxy task that trains VLMs to localize a subtle, synthetic visual hallucination injected into paragraphs of human-written image captions.<n>Models trained with the ViCrit Task exhibit substantial gains across a variety of vision-language models benchmarks.
  arXiv  Detail & Related papers  (2025-06-11T19:16:54Z)
• Visual Planning: Let's Think Only with Images [46.501897093584965]
  We argue that language may not always be the most natural or effective modality for reasoning, particularly in tasks involving spatial and geometrical information.<n>Motivated by this, we propose a new paradigm, Visual Planning, which enables planning through purely visual representations for these "vision-first" tasks.<n>In this paradigm, planning is executed via sequences of images that encode step-by-step inference in the visual domain, akin to how humans sketch or visualize future actions.
  arXiv  Detail & Related papers  (2025-05-16T16:17:22Z)
• Learning Concept-Based Causal Transition and Symbolic Reasoning for Visual Planning [36.131648635051334]
  Visual planning simulates how humans make decisions to achieve desired goals. We propose an interpretable and generalizable visual planning framework. We show that our framework can generalize to unseen task trajectories, unseen object categories, and real-world data.
  arXiv  Detail & Related papers  (2023-10-05T05:41:21Z)
• mPLUG: Effective and Efficient Vision-Language Learning by Cross-modal Skip-connections [104.14624185375897]
  mPLUG is a new vision-language foundation model for both cross-modal understanding and generation. It achieves state-of-the-art results on a wide range of vision-language downstream tasks, such as image captioning, image-text retrieval, visual grounding and visual question answering.
  arXiv  Detail & Related papers  (2022-05-24T11:52:06Z)
• SGEITL: Scene Graph Enhanced Image-Text Learning for Visual Commonsense Reasoning [61.57887011165744]
  multimodal Transformers have made great progress in the task of Visual Commonsense Reasoning. We propose a Scene Graph Enhanced Image-Text Learning framework to incorporate visual scene graphs in commonsense reasoning.
  arXiv  Detail & Related papers  (2021-12-16T03:16:30Z)
• Hallucinative Topological Memory for Zero-Shot Visual Planning [86.20780756832502]
  In visual planning (VP), an agent learns to plan goal-directed behavior from observations of a dynamical system obtained offline. Most previous works on VP approached the problem by planning in a learned latent space, resulting in low-quality visual plans. Here, we propose a simple VP method that plans directly in image space and displays competitive performance.
  arXiv  Detail & Related papers  (2020-02-27T18:54:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.