GEM: Generative Supervision Helps Embodied Intelligence

• URL: http://arxiv.org/abs/2605.28548v1
• Date: Wed, 27 May 2026 14:39:42 GMT
• Title: GEM: Generative Supervision Helps Embodied Intelligence
• Abstract summary: Embodied Vision-Language Models (VLMs) have demonstrated impressive performance and generalization in robotics.<n>A significant gap remains between the high-level semantic focus of standard text-guided pre-training paradigms and the low-level spatial and physical knowledge critical for execution in embodied environments.<n>We introduce GEM, a Generative-supervised Embodied vision-language Model designed to bridge this divide.
• Score: 60.28190143133428
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Embodied Vision-Language Models (VLMs) have demonstrated impressive performance and generalization in robotics, particularly within Vision-Language-Action frameworks. However, a significant gap remains between the high-level semantic focus of standard text-guided pre-training paradigms and the low-level spatial and physical knowledge critical for execution in embodied environments. In this paper, we introduce GEM, a Generative-supervised Embodied vision-language Model designed to bridge this divide. We propose integrating a depth map generation task directly into the VLM pre-training phase. By training this generative objective jointly with the main model, we observe substantial improvements in embodied intelligence, significantly enhancing both semantic understanding and physical operation capabilities. To support this paradigm, we curate and release GEM-4M, a comprehensive large-scale dataset featuring a mixture of grounding, reasoning, and planning data paired with high-quality depth supervision. Extensive experiments demonstrate that GEM achieves state-of-the-art results across diverse embodied benchmarks. Furthermore, our deployed action model, GEM-VLA, exhibits vastly superior task execution abilities in both simulation environments and real-world evaluations. Code, models, and datasets are available at https://zhaorw02.github.io/GEM/

Related papers

• Towards Generalizable Visually Grounded Exploration of Household Devices [74.66117923766632]
  Key bottleneck of realizing general autonomous embodied agents lies in Generalizable Visually Grounded Exploration.<n>We introduce VGEBench, a benchmark designed to evaluate the generalizable visually grounded exploration capabilities of Vision-Language Models.<n>Unlike static datasets, we construct a Logic-Driven State Machine framework. This framework simulates multi-turn interaction loops, compelling agents to achieve goals by active visual perception and feedback-driven correction.
  arXiv  Detail & Related papers  (2026-09-01T07:42:45Z)
• EPIC-Bench: A Perception-Centric Benchmark for Fine-Grained Embodied Visual Grounding in Vision-Language Models [39.77362541405281]
  EPIC-Bench is a grounding benchmark for large vision-language models (VLMs)<n>It systematically evaluates the visual perceptual capabilities of VLMs in real-world embodied environments.
  arXiv  Detail & Related papers  (2026-05-16T16:38:51Z)
• $M^2$-VLA: Boosting Vision-Language Models for Generalizable Manipulation via Layer Mixture and Meta-Skills [25.17452377052361]
  Current Vision-Language-Action (VLA) models predominantly rely on end-to-end fine-tuning.<n>We propose $M2$-VLA, which demonstrates that a generalized VLM is able to serve as a powerful backbone for robotic manipulation directly.<n>To overcome this, we introduce the Mixture of Layers (MoL) strategy that selectively extracts task-critical information from dense semantic features.
  arXiv  Detail & Related papers  (2026-04-27T08:44:12Z)
• Vision-and-Language Navigation for UAVs: Progress, Challenges, and a Research Roadmap [10.048113624715151]
  Vision-and-Language Navigation for Unmanned Aerial Vehicles (UAV-VLN) represents a pivotal challenge in embodied artificial intelligence.<n>This paper provides a comprehensive and structured survey of the field, from its formal task definition to the current state of the art.
  arXiv  Detail & Related papers  (2026-04-15T09:20:02Z)
• HY-Embodied-0.5: Embodied Foundation Models for Real-World Agents [71.09493646932046]
  We introduce HY-Embodied-0.5, a family of foundation models specifically designed for real-world embodied agents.<n>The HY-Embodied-0.5 suite comprises two primary variants: an efficient model with 2B activated parameters designed for edge deployment, and a powerful model with 32B activated parameters targeted for complex reasoning.<n>Our MoT-2B model outperforms similarly sized state-of-the-art models on 16 benchmarks, while the 32B variant achieves performance comparable to frontier models such as Gemini 3.0 Pro.
  arXiv  Detail & Related papers  (2026-04-08T17:59:48Z)
• DIAL: Decoupling Intent and Action via Latent World Modeling for End-to-End VLA [72.9197085473598]
  DIAL is a framework bridging high-level decision making and low-level motor execution through a differentiable latent intent bottleneck.<n>A VLM-based System-2 performs latent world modeling by latent visual foresight within the VLM's native feature space.<n>A lightweight System-1 policy then decodes this predicted intent together with the current observation into precise robot actions.
  arXiv  Detail & Related papers  (2026-03-31T15:02:27Z)
• SG-VLA: Learning Spatially-Grounded Vision-Language-Action Models for Mobile Manipulation [43.77409401429281]
  Vision-Language-Action (VLA) models show promise for robotic control, yet performance in complex household environments remains sub-optimal.<n>Mobile manipulation requires reasoning about global scene layout, fine-grained geometry, and high-dimensional continuous actions.<n>We introduce a framework for learning spatially-grounded VLA models that strengthens perception and representation.
  arXiv  Detail & Related papers  (2026-03-24T03:44:25Z)
• Revisiting Multi-Task Visual Representation Learning [52.93947931352643]
  We introduce MTV, a principled multi-task visual pretraining framework.<n>We leverage high-capacity "expert" models to synthesize dense, structured pseudo-labels at scale.<n>Our results demonstrate that MTV achieves "best-of-both-worlds" performance.
  arXiv  Detail & Related papers  (2026-01-20T11:59:19Z)
• Hierarchical Language Models for Semantic Navigation and Manipulation in an Aerial-Ground Robotic System [8.88014241557266]
  Heterogeneous multirobot systems show great potential in complex tasks requiring coordinated hybrid cooperation.<n>Existing methods that rely on static or task-specific models often lack generalizability across diverse tasks and dynamic environments.<n>We propose a hierarchical multimodal framework that integrates a prompted large language model (LLM) with a fine-tuned vision-language model (VLM)
  arXiv  Detail & Related papers  (2025-06-05T13:27:41Z)
• A Data-Centric Revisit of Pre-Trained Vision Models for Robot Learning [67.72413262980272]
  Pre-trained vision models (PVMs) are fundamental to modern robotics, yet their optimal configuration remains unclear.<n>We develop SlotMIM, a method that induces object-centric representations by introducing a semantic bottleneck.<n>Our approach achieves significant improvements over prior work in image recognition, scene understanding, and robot learning evaluations.
  arXiv  Detail & Related papers  (2025-03-10T06:18:31Z)
• ExGra-Med: Extended Context Graph Alignment for Medical Vision-Language Models [95.47808515575382]
  ExGra-Med is a novel framework for vision-language integration in medical AI.<n>It aligns images, instruction responses, and extended captions in the latent space, advancing semantic grounding and cross-modal coherence.<n>It matches LLaVA-Med's performance using just 10% of the pre-training data, achieving a 20.13% gain on VQA-RAD and approaching full-data performance.
  arXiv  Detail & Related papers  (2024-10-03T15:52:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.