FLARE: Robot Learning with Implicit World Modeling

• URL: http://arxiv.org/abs/2505.15659v1
• Date: Wed, 21 May 2025 15:33:27 GMT
• Title: FLARE: Robot Learning with Implicit World Modeling
• Abstract summary: $textbfFLARE$ integrates predictive latent world modeling into robot policy learning.<n>$textbfFLARE$ achieves state-of-the-art performance, outperforming prior policy learning baselines by up to 26%.<n>Our results establish $textbfFLARE$ as a general and scalable approach for combining implicit world modeling with high-frequency robotic control.
• Score: 87.81846091038676
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce $\textbf{F}$uture $\textbf{LA}$tent $\textbf{RE}$presentation Alignment ($\textbf{FLARE}$), a novel framework that integrates predictive latent world modeling into robot policy learning. By aligning features from a diffusion transformer with latent embeddings of future observations, $\textbf{FLARE}$ enables a diffusion transformer policy to anticipate latent representations of future observations, allowing it to reason about long-term consequences while generating actions. Remarkably lightweight, $\textbf{FLARE}$ requires only minimal architectural modifications -- adding a few tokens to standard vision-language-action (VLA) models -- yet delivers substantial performance gains. Across two challenging multitask simulation imitation learning benchmarks spanning single-arm and humanoid tabletop manipulation, $\textbf{FLARE}$ achieves state-of-the-art performance, outperforming prior policy learning baselines by up to 26%. Moreover, $\textbf{FLARE}$ unlocks the ability to co-train with human egocentric video demonstrations without action labels, significantly boosting policy generalization to a novel object with unseen geometry with as few as a single robot demonstration. Our results establish $\textbf{FLARE}$ as a general and scalable approach for combining implicit world modeling with high-frequency robotic control.

Related papers

• GaussianDream++: Efficient 3D Gaussian World Modeling for Robotic Manipulation [55.47130289185285]
  Vision-Language-Action (VLA) policies have advanced language-conditioned robotic manipulation.<n>We present textbfMethodname, a compact, policy-native extension that inserts textbfWorld State Tokens and textbfWorld Prediction Tokens directly into the VLA backbone.
  arXiv  Detail & Related papers  (2026-08-26T11:41:55Z)
• LeapBot-WA: World-Anchor Action Models via Predictive Latent Alignments [41.26480439505561]
  World Action Models (WAMs) have emerged as a powerful paradigm for embodied intelligence.<n>Forcing models to reconstruct task-irrelevant visual details dissipates representational capacity and renders policies vulnerable to visual distractors.<n>We propose LeapBot-WA, which establishes a novel Predictive-Latent paradigm for WAMs by operationalizing the Joint-Embedding Predictive Architecture (JEPA) as a World-Anchor.
  arXiv  Detail & Related papers  (2026-07-27T03:36:45Z)
• ReactVLA: Fast and Lightweight Reactive Robot Manipulation via Improved Mean Flow Action Generation [6.976481623762446]
  textttReactVLA is a lightweight and low-latency VLA framework for real-time robotic manipulation.<n>textttReactVLA consistently outperforms similarly sized VLA baselines.<n>It reduces real-world policy latency to below 38.6 ms, enabling fast reactive control on physical robot platforms.
  arXiv  Detail & Related papers  (2026-06-12T08:33:37Z)
• WEAVER, Better, Faster, Longer: An Effective World Model for Robotic Manipulation [26.48387646424999]
  WEAVER (World Estimation Across Views for Embodied Reasoning) is a WM architecture that simultaneously achieves all three desiderata.<n>We apply WEAVER in robotic hardware, demonstrating its effectiveness at policy evaluation, policy improvement, and test-time planning.
  arXiv  Detail & Related papers  (2026-06-11T17:59:15Z)
• $τ_0$-WM: A Unified Video-Action World Model for Robotic Manipulation [45.040666672458634]
  We present a unified video-action world model that integrates policy learning, video prediction, and action evaluation.<n>The model is trained on approximately $27,300$ hours of real-robot teleoperation.
  arXiv  Detail & Related papers  (2026-05-31T05:35:36Z)
• DexWorldModel: Causal Latent World Modeling towards Automated Learning of Embodied Tasks [54.32016216994156]
  We introduce the Causal Latent World Model (CLWM), which employs DINOv3 features as generative targets to disentangle interaction semantics from visual noise.<n>To overcome memory scaling, CLWM features a Dual-State Test-Time Training (TTT) Memory that guarantees a strict $mathcalO(1)$ footprint for long-horizon tasks.<n>We present EmbodiChain, an online framework that establishes the Efficiency Law by injecting an infinite flow of physics-grounded trajectories during training.
  arXiv  Detail & Related papers  (2026-04-13T03:19:36Z)
• Self-Improving World Modelling with Latent Actions [53.93276450137471]
  Internal modelling of the world is essential to reasoning and planning.<n>We propose SWIRL, a self-improvement framework that learns from state-only sequences.
  arXiv  Detail & Related papers  (2026-02-05T19:04:41Z)
• A4-Agent: An Agentic Framework for Zero-Shot Affordance Reasoning [29.682282730123234]
  Affordance prediction, which identifies interaction regions on objects based on language instructions, is critical for embodied AI.<n>We propose A4-Agent, a training-free agentic framework that decouples affordance prediction into a three-stage pipeline.<n>Our framework significantly outperforms state-of-the-art supervised methods across multiple benchmarks.
  arXiv  Detail & Related papers  (2025-12-16T14:27:47Z)
• H$^3$DP: Triply-Hierarchical Diffusion Policy for Visuomotor Learning [25.65324419553667]
  We introduce $textbfTriply-Hierarchical Diffusion Policy(textbfH$mathbf3$DP)$, a novel visuomotor learning framework that explicitly incorporates hierarchical structures to strengthen the integration between visual features and action generation.<n> Extensive experiments demonstrate that H$3$DP yields a $mathbf+27.5%$ average relative improvement over baselines across $mathbf44$ simulation tasks and achieves superior performance in $mathbf4$ challenging bimanual real-world manipulation tasks.
  arXiv  Detail & Related papers  (2025-05-12T17:59:43Z)
• $π_{0.5}$: a Vision-Language-Action Model with Open-World Generalization [81.73746512639283]
  We describe a new model based on $pi_0.5$ that uses co-training on heterogeneous tasks to enable broad generalization.<n>We demonstrate for the first time that an end-to-end learning-enabled robotic system can perform long-horizon and dexterous manipulation skills.
  arXiv  Detail & Related papers  (2025-04-22T17:31:29Z)
• HAMSTER: Hierarchical Action Models For Open-World Robot Manipulation [54.03004125910057]
  We show that hierarchical vision-language-action models can be more effective in utilizing off-domain data than standard monolithic VLA models.<n>We show that, with the hierarchical design, the high-level VLM can transfer across significant domain gaps between the off-domain finetuning data and real-robot testing scenarios.
  arXiv  Detail & Related papers  (2025-02-08T07:50:22Z)
• Robot Instance Segmentation with Few Annotations for Grasping [10.005879464111915]
  We propose a novel framework that combines Semi-Supervised Learning (SSL) with Learning Through Interaction (LTI)<n>Our approach exploits partially annotated data through self-supervision and incorporates temporal context using pseudo-sequences generated from unlabeled still images.<n>We validate our method on two common benchmarks, ARMBench mix-object-tote and OCID, where it achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2024-07-01T13:58:32Z)
• STORM: Efficient Stochastic Transformer based World Models for Reinforcement Learning [82.03481509373037]
  Recently, model-based reinforcement learning algorithms have demonstrated remarkable efficacy in visual input environments. We introduce Transformer-based wORld Model (STORM), an efficient world model architecture that combines strong modeling and generation capabilities. Storm achieves a mean human performance of $126.7%$ on the Atari $100$k benchmark, setting a new record among state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-10-14T16:42:02Z)
• Transferring Foundation Models for Generalizable Robotic Manipulation [82.12754319808197]
  We propose a novel paradigm that effectively leverages language-reasoning segmentation mask generated by internet-scale foundation models.<n>Our approach can effectively and robustly perceive object pose and enable sample-efficient generalization learning.<n>Demos can be found in our submitted video, and more comprehensive ones can be found in link1 or link2.
  arXiv  Detail & Related papers  (2023-06-09T07:22:12Z)
• Deep Imitation Learning for Bimanual Robotic Manipulation [70.56142804957187]
  We present a deep imitation learning framework for robotic bimanual manipulation. A core challenge is to generalize the manipulation skills to objects in different locations. We propose to (i) decompose the multi-modal dynamics into elemental movement primitives, (ii) parameterize each primitive using a recurrent graph neural network to capture interactions, and (iii) integrate a high-level planner that composes primitives sequentially and a low-level controller to combine primitive dynamics and inverse kinematics control.
  arXiv  Detail & Related papers  (2020-10-11T01:40:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.