Flex-$π$: A Multi-Stream World-Action Model with Compute Flexibility

• URL: http://arxiv.org/abs/2608.10860v2
• Date: Thu, 13 Aug 2026 07:46:45 GMT
• Title: Flex-$π$: A Multi-Stream World-Action Model with Compute Flexibility
• Abstract summary: We find a surprising free lunch: the same frozen video-generation VAE that encodes RGB also encodes 3D pointmaps almost losslessly, with no pointmap-specific training at all.<n>This lets us supervise Flex-$$, a 6B- parameter WAM, on 3D geometry and object-centric DINO semantics alongside RGB.<n>Every visual signal is projected into this shared latent space and denoised jointly with actions inside a Mixture-of-Transformers backbone.
• Score: 25.477428304143235
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: World-action models (WAMs) predict the future to act better, but nearly all of them predict only RGB latents, trained purely for pixel reconstruction, with no explicit signal for the 3D geometry or object semantics manipulation needs. We find a surprising free lunch: the same frozen video-generation VAE that encodes RGB also encodes 3D pointmaps almost losslessly, with no pointmap-specific training at all. This lets us supervise Flex-$π$, a 6B-parameter WAM, on 3D geometry and object-centric DINO semantics alongside RGB, at no cost in new sensors, new pre-training, or inference latency. Every visual signal is projected into this shared latent space and denoised jointly with actions inside a Mixture-of-Transformers backbone; per-stream dropout with cross-modality forcing then lets a single trained checkpoint run on any subset of these streams, from a fast action-only mode to full joint generation. The result is a policy that is exceptionally demonstration-efficient and generalizes well, beating the strongest baselines by up to 2-7$\times$ on dexterous, precise, real-world bimanual manipulation tasks both in and out of distribution, all while running faster than $π_{0.5}$. Our project website: https://flex-pi.github.io/

Related papers

• Disparity Has a Sign: Stereo Matching Beyond the Zero-Disparity Plane [52.210414985125816]
  We show how a stereoscopic content contains objects behind the zero-disparity plane corresponding to negative disparities.<n>We propose textitZDPShift, a benchmark of $21,495$ stereo pairs from seven cinematographer-authored open movies.<n>On identical scene content, Foundation goes from $2.24$ EPE to $75.33$px, with every backbone leaving roughly half of all pixels exceeding a three-pixel disparity error.
  arXiv  Detail & Related papers  (2026-09-06T19:58:35Z)
• GeoRay: Gauge-Aware Feed-Forward Satellite 3D Reconstruction in the Geodetic Frame [11.043507429399414]
  Method treats perspective-pretrained features, absolute elevation, and monocular and multi-view cues.<n> lightweight ray-consistent adapters make a frozen backbone matchable along native RPC rays.<n>bench, our absolute-frame benchmark of eighteen systems, scores absolute placement without registration or test-reference leakage.
  arXiv  Detail & Related papers  (2026-08-30T09:27:36Z)
• G$^3$VLA: Geometric inductive bias for Vision-Language-Action Models [3.704517635293094]
  Vision-language-action (VLA) models have made rapid progress in generalist robot manipulation.<n>Their visual tokens remain grounded in 2D image coordinates rather than the calibrated geometry of the robot's cameras.<n>We propose G$3$VLA, a camera-aware geometric module that injects calibrated structure into the visual-token stream of a pretrained VLA.
  arXiv  Detail & Related papers  (2026-06-23T12:02:36Z)
• $μ_0$: A Scalable 3D Interaction-Trace World Model [54.38434730157622]
  We present a scalable world model based on 3D traces.<n>Our TraceExtract system automatically extracts 3D supervision.<n>These results establish 3D traces as a scalable and transferable representation for cross-embodiment manipulation.
  arXiv  Detail & Related papers  (2026-06-11T17:59:56Z)
• Towards Camera-Robust 3D Localization: Equation-Anchored Tool-Use for MLLMs [72.8641426724502]
  3D localization in Multimodal Large Language Models (MLLMs) is limited by camera intrinsic ambiguity.<n>We propose an equation-anchored tool-use framework that re-purposes spatial tools as formula variables.<n>Our method outperforms RGB-only and tool-augmented baselines, with significant gains where the camera deviates most from the training scale.
  arXiv  Detail & Related papers  (2026-05-19T08:30:21Z)
• Action-Geometry Prediction with 3D Geometric Prior for Bimanual Manipulation [53.09168514034483]
  Bimanual manipulation requires policies that can reason about 3D geometry, anticipate how it evolves under action, and generate smooth, coordinated motions.<n>We propose a framework that builds bimanual manipulation directly on a pre-trained 3D geometric foundation model.<n>Our policy fuses geometry-aware latents, 2D semantic features, and proprioception into a unified state representation, and uses diffusion model to jointly predict a future action chunk and a future 3D latent that decodes into a dense pointmap.
  arXiv  Detail & Related papers  (2026-02-27T08:54:20Z)
• Test3R: Learning to Reconstruct 3D at Test Time [58.0912500917036]
  Test3R is a surprisingly simple test-time learning technique that significantly boosts geometric accuracy.<n>Our technique significantly outperforms previous state-of-the-art methods on the 3D reconstruction and multi-view depth estimation tasks.
  arXiv  Detail & Related papers  (2025-06-16T17:56:22Z)
• X-NeRF: Explicit Neural Radiance Field for Multi-Scene 360$^{\circ} $ Insufficient RGB-D Views [49.55319833743988]
  This paper focuses on a rarely discussed but important setting: can we train one model that can represent multiple scenes? We refer insufficient views to few extremely sparse and almost non-overlapping views. X-NeRF, a fully explicit approach which learns a general scene completion process instead of a coordinate-based mapping, is proposed.
  arXiv  Detail & Related papers  (2022-10-11T04:29:26Z)
• FIDNet: LiDAR Point Cloud Semantic Segmentation with Fully Interpolation Decoding [5.599306291149907]
  Projecting the point cloud on the 2D spherical range image transforms the LiDAR semantic segmentation to a 2D segmentation task on the range image. We propose a new projection-based LiDAR semantic segmentation pipeline that consists of a novel network structure and an efficient post-processing step. Our pipeline achieves the best performance among all projection-based methods with $64 times 2048$ resolution and all point-wise solutions.
  arXiv  Detail & Related papers  (2021-09-08T17:20:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.