Look, Zoom, Understand: The Robotic Eyeball for Embodied Perception

• URL: http://arxiv.org/abs/2511.15279v1
• Date: Wed, 19 Nov 2025 09:42:08 GMT
• Title: Look, Zoom, Understand: The Robotic Eyeball for Embodied Perception
• Abstract summary: Existing vision models and fixed RGB-D camera systems fail to reconcile wide-area coverage with fine-grained detail acquisition.<n>We propose EyeVLA, a robotic eyeball for active visual perception that can take proactive actions based on instructions.
• Score: 8.542874528320004
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In embodied AI perception systems, visual perception should be active: the goal is not to passively process static images, but to actively acquire more informative data within pixel and spatial budget constraints. Existing vision models and fixed RGB-D camera systems fundamentally fail to reconcile wide-area coverage with fine-grained detail acquisition, severely limiting their efficacy in open-world robotic applications. To address this issue, we propose EyeVLA, a robotic eyeball for active visual perception that can take proactive actions based on instructions, enabling clear observation of fine-grained target objects and detailed information across a wide spatial extent. EyeVLA discretizes action behaviors into action tokens and integrates them with vision-language models (VLMs) that possess strong open-world understanding capabilities, enabling joint modeling of vision, language, and actions within a single autoregressive sequence. By using the 2D bounding box coordinates to guide the reasoning chain and applying reinforcement learning to refine the viewpoint selection policy, we transfer the open-world scene understanding capability of the VLM to a vision language action (VLA) policy using only minimal real-world data. Experiments show that our system efficiently performs instructed scenes in real-world environments and actively acquires more accurate visual information through instruction-driven actions of rotation and zoom, thereby achieving strong environmental perception capabilities. EyeVLA introduces a novel robotic vision system that leverages detailed and spatially rich, large-scale embodied data, and actively acquires highly informative visual observations for downstream embodied tasks.

Related papers

• Afford-VLA: Action-Aligned Visual Planning via Internalized Affordance [108.46436073194546]
  We argue that effective planning should be local, visually grounded, internally generated, and directly aligned with action.<n>We propose Afford-VLA, a unified framework that internalizes task-conditioned affordance as an explicit visual planning interface within VLA models.
  arXiv  Detail & Related papers  (2026-05-22T20:43:47Z)
• FocusVLA: Focused Visual Utilization for Vision-Language-Action Models [12.859683124954339]
  Vision-Language-Action (VLA) models improve action generation by conditioning policies on rich vision-language information.<n>We introduce FocusVLA, a novel paradigm that directs the model's attention to task-relevant visual regions to effectively bridge vision to action.
  arXiv  Detail & Related papers  (2026-03-30T17:50:54Z)
• Look Before Acting: Enhancing Vision Foundation Representations for Vision-Language-Action Models [66.96421290733126]
  Vision-Language-Action (VLA) models have emerged as a promising paradigm for robotic manipulation.<n>We propose textbfDeepVision-VLA, built on a textbfVision-Language Mixture-of-Transformers (VL-MoT) framework.<n>DeepVision-VLA outperforms prior state-of-the-art methods by 9.0% and 7.5% on simulated and real-world tasks, respectively.
  arXiv  Detail & Related papers  (2026-03-16T17:59:54Z)
• I-Perceive: A Foundation Model for Active Perception with Language Instructions [41.67607728608853]
  We propose I-Perceive, a foundation model for active perception conditioned on natural language instructions.<n>I-Perceive predicts camera views that follow open-ended language instructions, based on image-based scene contexts.<n> Experiments demonstrate that I-Perceive significantly outperforms state-of-the-art VLMs in both prediction accuracy and instruction following of generated camera views.
  arXiv  Detail & Related papers  (2026-02-28T11:38:56Z)
• ActiveVLA: Injecting Active Perception into Vision-Language-Action Models for Precise 3D Robotic Manipulation [52.94334113271359]
  ActiveVLA is a vision-language-action framework that empowers robots with active perception capabilities for high-precision, fine-grained manipulation.<n>We show that ActiveVLA achieves precise 3D manipulation and outperforms state-of-the-art baselines on three simulation benchmarks.
  arXiv  Detail & Related papers  (2026-01-13T08:29:07Z)
• PhysVLM-AVR: Active Visual Reasoning for Multimodal Large Language Models in Physical Environments [36.84821207878773]
  Visual reasoning in multimodal large language models (MLLMs) has primarily been studied in static, fully observable settings.<n>We introduce the Active Visual Reasoning (AVR) task, extending visual reasoning to partially observable, interactive environments.<n>We present a benchmark featuring multi-round interactive environments designed to assess both reasoning and information-gathering efficiency.
  arXiv  Detail & Related papers  (2025-10-24T02:59:00Z)
• ReconVLA: Reconstructive Vision-Language-Action Model as Effective Robot Perceiver [35.25196177784228]
  We propose ReconVLA, a reconstructive VLA model with an implicit grounding paradigm.<n>Conditioned on the model's visual outputs, a diffusion transformer reconstructs the gaze region of the image.<n>This process prompts the VLA model to learn fine-grained representations and accurately allocate visual attention.
  arXiv  Detail & Related papers  (2025-08-14T04:20:19Z)
• Learning to See and Act: Task-Aware View Planning for Robotic Manipulation [88.37482534484627]
  Task-Aware View Planning (TAVP) is a framework designed to integrate active view planning with task-specific representation learning.<n>Our proposed TAVP model achieves superior performance over state-of-the-art fixed-view approaches.
  arXiv  Detail & Related papers  (2025-08-07T09:21:20Z)
• DreamVLA: A Vision-Language-Action Model Dreamed with Comprehensive World Knowledge [41.030494146004806]
  We propose DreamVLA, a novel VLA framework that integrates comprehensive world knowledge forecasting to enable inverse dynamics modeling.<n>DreamVLA introduces a dynamic-region-guided world knowledge prediction, integrated with the spatial and semantic cues, which provide compact yet comprehensive representations for action planning.<n>Experiments on both real-world and simulation environments demonstrate that DreamVLA achieves 76.7% success rate on real robot tasks.
  arXiv  Detail & Related papers  (2025-07-06T16:14:29Z)
• Flex: End-to-End Text-Instructed Visual Navigation from Foundation Model Features [59.892436892964376]
  We investigate the minimal data requirements and architectural adaptations necessary to achieve robust closed-loop performance with vision-based control policies.<n>Our findings are synthesized in Flex (Fly lexically), a framework that uses pre-trained Vision Language Models (VLMs) as frozen patch-wise feature extractors.<n>We demonstrate the effectiveness of this approach on a quadrotor fly-to-target task, where agents trained via behavior cloning successfully generalize to real-world scenes.
  arXiv  Detail & Related papers  (2024-10-16T19:59:31Z)
• LLaRA: Supercharging Robot Learning Data for Vision-Language Policy [56.505551117094534]
  We introduce LLaRA: Large Language and Robotics Assistant, a framework that formulates robot action policy as visuo-textual conversations.<n>First, we present an automated pipeline to generate conversation-style instruction tuning data for robots from existing behavior cloning datasets.<n>We show that a VLM finetuned with a limited amount of such datasets can produce meaningful action decisions for robotic control.
  arXiv  Detail & Related papers  (2024-06-28T17:59:12Z)
• Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs [61.143381152739046]
  We introduce Cambrian-1, a family of multimodal LLMs (MLLMs) designed with a vision-centric approach.<n>Our study uses LLMs and visual instruction tuning as an interface to evaluate various visual representations.<n>We provide model weights, code, supporting tools, datasets, and detailed instruction-tuning and evaluation recipes.
  arXiv  Detail & Related papers  (2024-06-24T17:59:42Z)
• MOKA: Open-World Robotic Manipulation through Mark-Based Visual Prompting [97.52388851329667]
  We introduce Marking Open-world Keypoint Affordances (MOKA) to solve robotic manipulation tasks specified by free-form language instructions. Central to our approach is a compact point-based representation of affordance, which bridges the VLM's predictions on observed images and the robot's actions in the physical world. We evaluate and analyze MOKA's performance on various table-top manipulation tasks including tool use, deformable body manipulation, and object rearrangement.
  arXiv  Detail & Related papers  (2024-03-05T18:08:45Z)
• Voila-A: Aligning Vision-Language Models with User's Gaze Attention [56.755993500556734]
  We introduce gaze information as a proxy for human attention to guide Vision-Language Models (VLMs) We propose a novel approach, Voila-A, for gaze alignment to enhance the interpretability and effectiveness of these models in real-world applications.
  arXiv  Detail & Related papers  (2023-12-22T17:34:01Z)
• Look Before You Leap: Unveiling the Power of GPT-4V in Robotic Vision-Language Planning [32.045840007623276]
  We introduce Robotic Vision-Language Planning (ViLa), a novel approach for long-horizon robotic planning. ViLa directly integrates perceptual data into its reasoning and planning process. Our evaluation, conducted in both real-robot and simulated environments, demonstrates ViLa's superiority over existing LLM-based planners.
  arXiv  Detail & Related papers  (2023-11-29T17:46:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.