VCA: Vision-Click-Action Framework for Precise Manipulation of Segmented Objects in Target Ambiguous Environments

• URL: http://arxiv.org/abs/2602.23583v1
• Date: Fri, 27 Feb 2026 01:23:35 GMT
• Title: VCA: Vision-Click-Action Framework for Precise Manipulation of Segmented Objects in Target Ambiguous Environments
• Abstract summary: We propose Vision-Click-Action (VCA), a framework that replaces textual commands with direct, click-based visual interaction.<n>VCA reduces interpretation errors, lowers cognitive load, and provides a practical and scalable alternative to language-driven interfaces for real-world robotic manipulation.
• Score: 2.566571621858397
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: The reliance on language in Vision-Language-Action (VLA) models introduces ambiguity, cognitive overhead, and difficulties in precise object identification and sequential task execution, particularly in environments with multiple visually similar objects. To address these limitations, we propose Vision-Click-Action (VCA), a framework that replaces verbose textual commands with direct, click-based visual interaction using pretrained segmentation models. By allowing operators to specify target objects clearly through visual selection in the robot's 2D camera view, VCA reduces interpretation errors, lowers cognitive load, and provides a practical and scalable alternative to language-driven interfaces for real-world robotic manipulation. Experimental results validate that the proposed VCA framework achieves effective instance-level manipulation of specified target objects. Experiment videos are available at https://robrosinc.github.io/vca/.

Related papers

• GesVLA: Gesture-Aware Vision-Language-Action Model Embedded Representations [19.347332945780273]
  Vision-Language-Action (VLA) models have shown strong potential for general-purpose robot manipulation by unifying perception and action.<n>We introduce gesture as a parallel instruction modality and propose a Gesture-aware Vision-Language-Action model (GesVLA)<n> Experimental results show that incorporating gesture consistently improves target grounding accuracy and human-robot interaction efficiency.
  arXiv  Detail & Related papers  (2026-05-21T17:57:44Z)
• Bring My Cup! Personalizing Vision-Language-Action Models with Visual Attentive Prompting [15.173629384871013]
  Visual Attentive Prompting (VAP) is a training-free perceptual adapter that equips frozen Vision-Language-Action models with selective attention.<n>VAP consistently outperforms generic policies and token-learning baselines in both success rate and correct-object manipulation.
  arXiv  Detail & Related papers  (2025-12-23T03:13:39Z)
• Improving Generalization of Language-Conditioned Robot Manipulation [29.405161073483175]
  We present a framework that learns object-arrangement tasks from just a few demonstrations.<n>We validate our method on both simulation and real-world robotic environments.
  arXiv  Detail & Related papers  (2025-08-04T13:29:26Z)
• InterRVOS: Interaction-aware Referring Video Object Segmentation [44.55538737075162]
  We introduce Interaction-aware Referring Video Object (InterRVOS), a novel task that focuses on the modeling of interactions.<n>It requires the model to segment the actor and target objects separately, reflecting their asymmetric roles in an interaction.<n>We present InterRVOS-127K, a large-scale dataset with over 127K automatically annotated expressions, including interaction expressions annotated with distinct masks for actor and target objects.
  arXiv  Detail & Related papers  (2025-06-03T01:16:13Z)
• CTRL-O: Language-Controllable Object-Centric Visual Representation Learning [30.218743514199016]
  Object-centric representation learning aims to decompose visual scenes into fixed-size vectors called "slots" or "object files"<n>Current object-centric models learn representations based on their preconceived understanding of objects, without allowing user input to guide which objects are represented.<n>We propose a novel approach for user-directed control over slot representations by conditioning slots on language descriptions.
  arXiv  Detail & Related papers  (2025-03-27T17:53:50Z)
• OTTER: A Vision-Language-Action Model with Text-Aware Visual Feature Extraction [95.6266030753644]
  Vision-Language-Action (VLA) models aim to predict robotic actions based on visual observations and language instructions.<n>Existing approaches require fine-tuning pre-trained vision-language models (VLMs) as visual and language features are independently fed into downstream policies.<n>We propose OTTER, a novel VLA architecture that leverages existing alignments through explicit, text-aware visual feature extraction.
  arXiv  Detail & Related papers  (2025-03-05T18:44:48Z)
• Teaching VLMs to Localize Specific Objects from In-context Examples [56.797110842152]
  We find that present-day Vision-Language Models (VLMs) lack a fundamental cognitive ability: learning to localize specific objects in a scene by taking into account the context.<n>This work is the first to explore and benchmark personalized few-shot localization for VLMs.
  arXiv  Detail & Related papers  (2024-11-20T13:34:22Z)
• Position-Aware Contrastive Alignment for Referring Image Segmentation [65.16214741785633]
  We present a position-aware contrastive alignment network (PCAN) to enhance the alignment of multi-modal features. Our PCAN consists of two modules: 1) Position Aware Module (PAM), which provides position information of all objects related to natural language descriptions, and 2) Contrastive Language Understanding Module (CLUM), which enhances multi-modal alignment.
  arXiv  Detail & Related papers  (2022-12-27T09:13:19Z)
• Semantically Grounded Object Matching for Robust Robotic Scene Rearrangement [21.736603698556042]
  We present a novel approach to object matching that uses a large pre-trained vision-language model to match objects in a cross-instance setting. We demonstrate that this provides considerably improved matching performance in cross-instance settings.
  arXiv  Detail & Related papers  (2021-11-15T18:39:43Z)
• INVIGORATE: Interactive Visual Grounding and Grasping in Clutter [56.00554240240515]
  INVIGORATE is a robot system that interacts with human through natural language and grasps a specified object in clutter. We train separate neural networks for object detection, for visual grounding, for question generation, and for OBR detection and grasping. We build a partially observable Markov decision process (POMDP) that integrates the learned neural network modules.
  arXiv  Detail & Related papers  (2021-08-25T07:35:21Z)
• Rethinking Cross-modal Interaction from a Top-down Perspective for Referring Video Object Segmentation [140.4291169276062]
  Referring video object segmentation (RVOS) aims to segment video objects with the guidance of natural language reference. Previous methods typically tackle RVOS through directly grounding linguistic reference over the image lattice. In this work, we put forward a two-stage, top-down RVOS solution. First, an exhaustive set of object tracklets is constructed by propagating object masks detected from several sampled frames to the entire video. Second, a Transformer-based tracklet-language grounding module is proposed, which models instance-level visual relations and cross-modal interactions simultaneously and efficiently.
  arXiv  Detail & Related papers  (2021-06-02T10:26:13Z)
• Object-and-Action Aware Model for Visual Language Navigation [70.33142095637515]
  Vision-and-Language Navigation (VLN) is unique in that it requires turning relatively general natural-language instructions into robot agent actions. We propose an Object-and-Action Aware Model (OAAM) that processes these two different forms of natural language based instruction separately. This enables each process to match object-centered/action-centered instruction to their own counterpart visual perception/action orientation flexibly.
  arXiv  Detail & Related papers  (2020-07-29T06:32:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.