Scaling Mixture-of-Experts Video Pretraining for Embodied Intelligence

• URL: http://arxiv.org/abs/2607.07675v1
• Date: Wed, 08 Jul 2026 17:33:40 GMT
• Title: Scaling Mixture-of-Experts Video Pretraining for Embodied Intelligence
• Abstract summary: LingBot-Video is a DiT-based video pretraining paradigm specifically tailored for embodied intelligence.<n>From the data perspective, we construct a data profiling engine that augments standard internet videos with extensive robot-oriented footage.<n>From the training perspective, we develop a multi-dimensional reward system to enforce the alignment regarding physical rationality and task completion.
• Score: 73.39562379177796
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Despite the recent promise in robot control, video generative models suffer from a domain mismatch due to their primary focus on content creation. For example, their design inherently prioritizes visual fidelity and creativity over computational efficiency and physical realism. In this work, we present LingBot-Video, a DiT-based video pretraining paradigm specifically tailored for embodied intelligence. From the architecture perspective, we adopt the Mixture-of-Experts (MoE), instead of dense, framework to achieve a better trade-off between modeling capacity and inference efficiency, and manage to scale it up from scratch. From the data perspective, we construct a data profiling engine that augments standard internet videos with extensive robot-oriented footage, encompassing manipulation, navigation, and egocentric perspectives, to equip the base model with an intrinsic understanding of actions and world dynamics. From the training perspective, we develop a multi-dimensional reward system to enforce the alignment regarding physical rationality and task completion, going beyond standard criteria such as aesthetics, prompt-following, and motion consistency. Comprehensive evaluations validate its performance and efficiency as a video foundation model. We contribute LingBot-Video as the inaugural large-scale, open-source MoE video foundation model to the community, in a pioneering effort to bridge digital creativity and physical actuation.

Related papers

• Building a Precise Video Language with Human-AI Oversight [63.57293532815199]
  CHAI (Critique-based Human-AI Oversight) is a framework where trained experts critique and revise model-language pre-captions into improved post-captions.<n>Our ablations show that critique quality in precision, recall, and constructiveness, ensured by our oversight framework, directly governs downstream performance.
  arXiv  Detail & Related papers  (2026-04-22T09:01:04Z)
• DreamDojo: A Generalist Robot World Model from Large-Scale Human Videos [110.98100817695307]
  We introduce DreamDojo, a foundation world model that learns diverse interactions and dexterous controls from 44k hours of egocentric human videos.<n>Our work enables several important applications based on generative world models, including live teleoperation, policy evaluation, and model-based planning.
  arXiv  Detail & Related papers  (2026-02-06T18:49:43Z)
• ReWorld: Multi-Dimensional Reward Modeling for Embodied World Models [27.729654985554372]
  ReWorld is a framework aimed to employ reinforcement learning to align the video-based embodied world models with physical realism, task completion capability, embodiment plausibility and visual quality.<n>We show that ReWorld significantly improves the physical fidelity, logical coherence, embodiment and visual quality of generated rollouts, outperforming previous methods.
  arXiv  Detail & Related papers  (2026-01-18T14:27:10Z)
• mimic-video: Video-Action Models for Generalizable Robot Control Beyond VLAs [5.109732854501585]
  We introduce mimic-video, a novel Video-Action Model (VAM) that pairs a pretrained Internet-scale video model with a flow matching-based action decoder conditioned on its latent representations.<n>Our approach achieves state-of-the-art performance on simulated and real-world robotic manipulation tasks, improving sample efficiency by 10x and convergence speed by 2x compared to traditional VLA architectures.
  arXiv  Detail & Related papers  (2025-12-17T18:47:31Z)
• Large Video Planner Enables Generalizable Robot Control [117.49024534548319]
  General-purpose robots require decision-making models that generalize across diverse tasks and environments.<n>Recent works build robot foundation models by extending multimodal large language models (LMs) with action outputs, creating vision--action (VLA) systems.<n>We explore an alternative paradigm of using large-scale video pretraining as a primary modality for building robot foundation models.
  arXiv  Detail & Related papers  (2025-12-17T18:35:54Z)
• Simulating the Visual World with Artificial Intelligence: A Roadmap [48.64639618440864]
  Video generation is shifting from generating visually appealing clips to building virtual environments that support interaction and maintain physical plausibility.<n>This survey provides a systematic overview of this evolution, conceptualizing modern video foundation models as the combination of two core components.<n>We trace the progression of video generation through four generations, culminating in a video generation model that embodies intrinsic physical plausibility.
  arXiv  Detail & Related papers  (2025-11-11T18:59:50Z)
• RoboScape: Physics-informed Embodied World Model [25.61586473778092]
  We present RoboScape, a unified physics-informed world model that jointly learns RGB video generation and physics knowledge.<n>Experiments demonstrate that RoboScape generates videos with superior visual fidelity and physical plausibility across diverse robotic scenarios.<n>Our work provides new insights for building efficient physics-informed world models to advance embodied intelligence research.
  arXiv  Detail & Related papers  (2025-06-29T08:19:45Z)
• LensCraft: Your Professional Virtual Cinematographer [12.512681517449868]
  Digital creators, from indie filmmakers to animation studios, face a persistent bottleneck: translating their creative vision into precise camera movements.<n>LensCraft solves this problem by mimicking the expertise of a professional cinematographer, using a data-driven approach.<n>LensCraft achieves markedly lower computational complexity and faster inference while maintaining high output quality.
  arXiv  Detail & Related papers  (2025-06-01T12:43:55Z)
• Masked World Models for Visual Control [90.13638482124567]
  We introduce a visual model-based RL framework that decouples visual representation learning and dynamics learning. We demonstrate that our approach achieves state-of-the-art performance on a variety of visual robotic tasks.
  arXiv  Detail & Related papers  (2022-06-28T18:42:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.