NORA: A Small Open-Sourced Generalist Vision Language Action Model for Embodied Tasks

• URL: http://arxiv.org/abs/2504.19854v1
• Date: Mon, 28 Apr 2025 14:47:34 GMT
• Title: NORA: A Small Open-Sourced Generalist Vision Language Action Model for Embodied Tasks
• Authors: Chia-Yu Hung, Qi Sun, Pengfei Hong, Amir Zadeh, Chuan Li, U-Xuan Tan, Navonil Majumder, Soujanya Poria,
• Abstract summary: Existing Visual-Language-Action (VLA) models have shown promising performance in zero-shot scenarios.<n>These models typically suffer from high computational overhead due to their large sizes.<n>We propose NORA, a model designed to reduce computational overhead while maintaining strong task performance.
• Score: 37.03331507197761
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Existing Visual-Language-Action (VLA) models have shown promising performance in zero-shot scenarios, demonstrating impressive task execution and reasoning capabilities. However, a significant challenge arises from the limitations of visual encoding, which can result in failures during tasks such as object grasping. Moreover, these models typically suffer from high computational overhead due to their large sizes, often exceeding 7B parameters. While these models excel in reasoning and task planning, the substantial computational overhead they incur makes them impractical for real-time robotic environments, where speed and efficiency are paramount. To address the limitations of existing VLA models, we propose NORA, a 3B-parameter model designed to reduce computational overhead while maintaining strong task performance. NORA adopts the Qwen-2.5-VL-3B multimodal model as its backbone, leveraging its superior visual-semantic understanding to enhance visual reasoning and action grounding. Additionally, our \model{} is trained on 970k real-world robot demonstrations and equipped with the FAST+ tokenizer for efficient action sequence generation. Experimental results demonstrate that NORA outperforms existing large-scale VLA models, achieving better task performance with significantly reduced computational overhead, making it a more practical solution for real-time robotic autonomy.

Related papers

• CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models [89.44024245194315]
  We introduce a method that incorporates explicit visual chain-of-thought (CoT) reasoning into vision-language-action models (VLAs) We introduce CoT-VLA, a state-of-the-art 7B VLA that can understand and generate visual and action tokens. Our experimental results demonstrate that CoT-VLA achieves strong performance, outperforming the state-of-the-art VLA model by 17% in real-world manipulation tasks and 6% in simulation benchmarks.
  arXiv  Detail & Related papers  (2025-03-27T22:23:04Z)
• TraceVLA: Visual Trace Prompting Enhances Spatial-Temporal Awareness for Generalist Robotic Policies [95.30717188630432]
  We introduce visual trace prompting to facilitate VLA models' spatial-temporal awareness for action prediction. We develop a new TraceVLA model by finetuning OpenVLA on our own collected dataset of 150K robot manipulation trajectories. We present a compact VLA model based on 4B Phi-3-Vision, pretrained on the Open-X-Embodiment and finetuned on our dataset.
  arXiv  Detail & Related papers  (2024-12-13T18:40:51Z)
• CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulation [100.25567121604382]
  Vision-Language-Action (VLA) models have improved robotic manipulation in terms of language-guided task execution and generalization to unseen scenarios.<n>We present a new advanced VLA architecture derived from Vision-Language-Models (VLM)<n>We show that our model not only significantly surpasses existing VLAs in task performance and but also exhibits remarkable adaptation to new robots and generalization to unseen objects and backgrounds.
  arXiv  Detail & Related papers  (2024-11-29T12:06:03Z)
• A Dual Process VLA: Efficient Robotic Manipulation Leveraging VLM [0.26334346517416873]
  Vision-Language-Action (VLA) models enable robots to perform complex tasks by integrating visual context with linguistic commands. To overcome this, we propose Dual Process VLA (DP-VLA), a hierarchical framework inspired by dual-process theory. Experimental results on the RoboCasa dataset demonstrate that DP-VLA achieves faster inference and higher task success rates.
  arXiv  Detail & Related papers  (2024-10-21T00:36:02Z)
• Run-time Observation Interventions Make Vision-Language-Action Models More Visually Robust [9.647148940880381]
  Vision-language-action (VLA) models trained on large-scale internet data and robot demonstrations have the potential to serve as generalist robot policies. We introduce Bring Your Own VLA (BYOVLA): a run-time intervention scheme that dynamically identifies regions of the input image that the model is sensitive to. We show that BYOVLA enables state-of-the-art VLA models to nearly retain their nominal performance in the presence of distractor objects and backgrounds.
  arXiv  Detail & Related papers  (2024-10-02T19:29:24Z)
• VIRL: Volume-Informed Representation Learning towards Few-shot Manufacturability Estimation [0.0]
  This work introduces VIRL, a Volume-Informed Representation Learning approach to pre-train a 3D geometric encoder. The model pre-trained by VIRL shows substantial enhancements on demonstrating improved generalizability with limited data.
  arXiv  Detail & Related papers  (2024-06-18T05:30:26Z)
• SAM-E: Leveraging Visual Foundation Model with Sequence Imitation for Embodied Manipulation [62.58480650443393]
  Segment Anything (SAM) is a vision-foundation model for generalizable scene understanding and sequence imitation. We develop a novel multi-channel heatmap that enables the prediction of the action sequence in a single pass.
  arXiv  Detail & Related papers  (2024-05-30T00:32:51Z)
• Data-efficient Large Vision Models through Sequential Autoregression [58.26179273091461]
  We develop an efficient, autoregression-based vision model on a limited dataset. We demonstrate how this model achieves proficiency in a spectrum of visual tasks spanning both high-level and low-level semantic understanding. Our empirical evaluations underscore the model's agility in adapting to various tasks, heralding a significant reduction in the parameter footprint.
  arXiv  Detail & Related papers  (2024-02-07T13:41:53Z)
• BYOM: Building Your Own Multi-Task Model For Free [69.63765907216442]
  BYOM-FFT is for merging fully finetuned models, while BYOM-LoRA is for LoRA-finetuned models. Experiments on computer vision and natural language processing tasks show that the proposed BYOM methods outperform existing merging methods by a large margin.
  arXiv  Detail & Related papers  (2023-10-03T08:39:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.