Modeling What Changes: Sparse, Residual World Models for Object-Centric Manipulation

• URL: http://arxiv.org/abs/2609.02046v1
• Date: Wed, 02 Sep 2026 03:20:09 GMT
• Title: Modeling What Changes: Sparse, Residual World Models for Object-Centric Manipulation
• Abstract summary: We show that the sparse/residual model predicts next-state poses 2.5 to 4.6 times more accurately than a dense multilayer perceptron.<n>In autoregressive rollout it compounds far less error, hugging the no-motion floor while the dense model drifts.<n> Modeling what changes, rather than re-predicting the whole world, is a simple, effective bias for object-centric physical AI.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Monolithic world models predict the entire next state at every step, spending capacity re-predicting the static majority of a scene and injecting error into it. We ask whether explicitly modeling change (a per-object change gate plus a residual delta head that perturbs only the objects the gate flags) is a more effective and interpretable bias for physical prediction and control. On a MuJoCo tabletop pushing benchmark scaling from 3 to 8 objects, the sparse/residual model predicts next-state poses 2.5 to 4.6 times more accurately than a dense multilayer perceptron at 8.6 to 11.1 times fewer parameters, sustains change-detection F1 of 0.80 to 0.87 where the dense baseline is degenerate, transfers across object counts with zero retraining (99.4 percent F1 retention), and reaches about 90 percent of its full-data accuracy with a quarter of the data. In autoregressive rollout it compounds far less error, hugging the no-motion floor while the dense model drifts. Finally, inside a sampling-based planner, prediction-only models fail (though a true-simulator oracle solves the task with the identical planner, confirming the planner is sound), but once featurized and trained for the states a planner visits, the sparse model begins to plan (0.23 plus or minus 0.06 success over three seeds) while the dense monolith stays at zero at every seed. Modeling what changes, rather than re-predicting the whole world, is a simple, effective bias for object-centric physical AI; code, data generators, and all checkpoints will be released upon publication.

Related papers

• Teaching Vision-Language Models to Use the Scale They Are Given: Label-Free Equivariance Training for Metric Physical Reasoning [15.015602453377516]
  Metric questions about video require vision-language models to convert visual measurements into physical units.<n>We find that current models use this scale information only partially.<n>Across eight vision-language models, this under-response persists over four orders of magnitude.<n>We use this exact scaling relation as supervision without requiring metric annotations.
  arXiv  Detail & Related papers  (2026-09-01T03:36:33Z)
• Distribution-First Population Simulation: Collapse, Calibration, and Recall in Non-WEIRD LLM Persona Modeling [0.0]
  Synthetic-population tools run every individual as an independent large language model (LLM) agent.<n>We show that this paradigm has a basic failure mode, and we set a distribution-first corrective against it.
  arXiv  Detail & Related papers  (2026-07-17T15:45:57Z)
• From Surface Forecasting to Observability Forecasting: A Latent World Model for Cloud-Aware EO Monitoring [0.0]
  We adapt LeWorldModel, a joint-embedding predictive architecture world model, to cloud-aware Earth Observation sequences.<n>The resulting model has 18.0M trainable parameters and is trained from scratch on 23,904 training episodes.<n>LeWorldModel consistently outperforms persistence on the full frozen-bundle observability benchmark.
  arXiv  Detail & Related papers  (2026-07-15T09:52:22Z)
• PatchWorld: Gradient-Free Optimization of Executable World Models [69.05094454437402]
  We introduce PatchWorld, a gradient-free framework that turns offline trajectories into executable Python world models.<n>Across seven AgentGym environments, PatchWorld-Simple achieves the highest code-based planning score.<n>We find that a human-specified residual-memory bias improves surface observation fidelity but weakens decision utility.
  arXiv  Detail & Related papers  (2026-05-29T06:11:14Z)
• Articulate but Wrong: Self-Review Failures in LLM-Based Code Modernization [1.0164694825170502]
  Large language model (LLM) agents are increasingly used to migrate legacy code to modern stacks.<n>We run 1,980 real modernization calls across 11 production LLMs from 7 distinct families on a balanced 60-snippet legacy-Python-2 corpus.<n>We find that semantic-preservation drift is prevalent and sharply separable from a cleanly-controlled baseline.
  arXiv  Detail & Related papers  (2026-05-20T05:00:31Z)
• Solve the Loop: Attractor Models for Language and Reasoning [4.8720589853137435]
  Looped Transformers offer a promising alternative to purely feed-forward computation.<n>We introduce Attractor Models, in which a backbone module first proposes output embeddings, then an attractor module refines them by solving for the fixed point.<n>We show that Attractor Models outperform existing models across two regimes, large-scale language-model pretraining and reasoning with tiny models.
  arXiv  Detail & Related papers  (2026-05-12T17:51:26Z)
• LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels [49.35636088613484]
  Joint Embedding Predictive Architectures (JEPAs) offer a compelling framework for learning world models in compact latent spaces.<n>We introduce LeModelWorld, the first JEPA that trains stably end-to-end from raw pixels using only two loss terms.<n>With 15M parameters trainable on a single GPU in a few hours, LeWM plans up to 48x faster than foundation-model-based world models.
  arXiv  Detail & Related papers  (2026-03-13T19:48:14Z)
• Agentic World Modeling for 6G: Near-Real-Time Generative State-Space Reasoning [70.56067503630486]
  We argue that sixth-generation (6G) intelligence is not fluent token prediction but calibrated the capacity to imagine and choose.<n>We show that WM-MS3M cuts mean absolute error (MAE) by 1.69% versus MS3M with 32% fewer parameters and similar latency, and achieves 35-80% lower root mean squared error (RMSE) than attention/hybrid baselines with 2.3-4.1x faster inference.
  arXiv  Detail & Related papers  (2025-11-04T17:22:22Z)
• DataDecide: How to Predict Best Pretraining Data with Small Experiments [67.95896457895404]
  We release models, data, and evaluations in DataDecide -- the most extensive open suite of models over differences in data and scale.<n>We conduct controlled pretraining experiments across 25 corpora with differing sources, deduplication, and filtering up to 100B tokens, model sizes up to 1B parameters, and 3 random seeds.
  arXiv  Detail & Related papers  (2025-04-15T17:02:15Z)
• WorldModelBench: Judging Video Generation Models As World Models [57.776769550453594]
  Video generation models have rapidly progressed, positioning themselves as video world models capable of supporting decision-making applications like robotics and autonomous driving.<n>Current benchmarks fail to rigorously evaluate these claims, focusing only on general video quality.<n>We propose WorldModelBench, a benchmark designed to evaluate the world modeling capabilities of video generation models in application-driven domains.
  arXiv  Detail & Related papers  (2025-02-28T03:58:23Z)
• Mismatched No More: Joint Model-Policy Optimization for Model-Based RL [172.37829823752364]
  We propose a single objective for jointly training the model and the policy, such that updates to either component increases a lower bound on expected return. Our objective is a global lower bound on expected return, and this bound becomes tight under certain assumptions. The resulting algorithm (MnM) is conceptually similar to a GAN.
  arXiv  Detail & Related papers  (2021-10-06T13:43:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.