CompassAD: Intent-Driven 3D Affordance Grounding in Functionally Competing Objects

• URL: http://arxiv.org/abs/2604.02060v1
• Date: Thu, 02 Apr 2026 13:57:01 GMT
• Title: CompassAD: Intent-Driven 3D Affordance Grounding in Functionally Competing Objects
• Abstract summary: In real-world scenes, multiple objects may share identical affordances, yet only one is appropriate under the given task context.<n>We formalize Multi-Object Affordance Grounding under Intent-Driven Instructions.<n>We construct CompassAD, the first benchmark centered on implicit intent in confusable multi-object scenes.
• Score: 23.801781337960914
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: When told to "cut the apple," a robot must choose the knife over nearby scissors, despite both objects affording the same cutting function. In real-world scenes, multiple objects may share identical affordances, yet only one is appropriate under the given task context. We call such cases confusing pairs. However, existing 3D affordance methods largely sidestep this challenge by evaluating isolated single objects, often with explicit category names provided in the query. We formalize Multi-Object Affordance Grounding under Intent-Driven Instructions, a new 3D affordance setting that requires predicting a per-point affordance mask on the correct object within a cluttered multi-object point cloud, conditioned on implicit natural language intent. To study this problem, we construct CompassAD, the first benchmark centered on implicit intent in confusable multi-object scenes. It comprises 30 confusing object pairs spanning 16 affordance types, 6,422 scenes, and 88K+ query-answer pairs. Furthermore, we propose CompassNet, a framework that incorporates two dedicated modules tailored to this task. Instance-bounded Cross Injection (ICI) constrains language-geometry alignment within object boundaries to prevent cross-object semantic leakage. Bi-level Contrastive Refinement (BCR) enforces discrimination at both geometric-group and point levels, sharpening distinctions between target and confusable surfaces. Extensive experiments demonstrate state-of-the-art results on both seen and unseen queries, and deployment on a robotic manipulator confirms effective transfer to real-world grasping in confusing multi-object scenes.

Related papers

• SOS! : A Streamlined Object-Conditional Transformer for Model-free Segmentation [67.361660179434]
  We propose SOS (Streamlined Object-conditional Transformer for model-free pipelines) for unseen objects segmentation.<n>Central to our framework is a novel Object-Conditional Transformer that learns identity-anchored queries, unifying mask generation and target identification into a single feed-forward pass.<n>Sos establishes a new state-of-the-art for model-free unseen objects segmentation, delivering accurate and high-efficiency performance.
  arXiv  Detail & Related papers  (2026-08-15T15:53:23Z)
• SaaF: Scene-Specific Ambiguity-Aware 3D Language Fields towards Interactive Real-World Object Retrieval [3.5163697824409295]
  Scene-specific Ambiguity-aware 3D Language Fields (SaaF)<n>We propose a novel 3D language field designed for interactive object retrieval in a given real-world scene.<n> Experimental results demonstrate that SaaF not only improves retrieval accuracy over previous methods but also robustly detects and handles ambiguity in the user text queries under open-vocabulary settings.
  arXiv  Detail & Related papers  (2026-07-14T15:32:10Z)
• LESS: Label-Efficient and Single-Stage Referring 3D Segmentation [55.06002976797879]
  Referring 3D is a visual-language task that segments all points of the specified object from a 3D point cloud described by a sentence of query. We propose a novel Referring 3D pipeline, Label-Efficient and Single-Stage, dubbed LESS, which is only under the supervision of efficient binary mask. We achieve state-of-the-art performance on ScanRefer dataset by surpassing the previous methods about 3.7% mIoU using only binary labels.
  arXiv  Detail & Related papers  (2024-10-17T07:47:41Z)
• DOCTR: Disentangled Object-Centric Transformer for Point Scene Understanding [7.470587868134298]
  Point scene understanding is a challenging task to process real-world scene point cloud. Recent state-of-the-art method first segments each object and then processes them independently with multiple stages for the different sub-tasks. We propose a novel Disentangled Object-Centric TRansformer (DOCTR) that explores object-centric representation.
  arXiv  Detail & Related papers  (2024-03-25T05:22:34Z)
• Chat-Scene: Bridging 3D Scene and Large Language Models with Object Identifiers [65.51132104404051]
  We introduce the use of object identifiers and object-centric representations to interact with scenes at the object level. Our model significantly outperforms existing methods on benchmarks including ScanRefer, Multi3DRefer, Scan2Cap, ScanQA, and SQA3D.
  arXiv  Detail & Related papers  (2023-12-13T14:27:45Z)
• Object-Centric Multiple Object Tracking [124.30650395969126]
  This paper proposes a video object-centric model for multiple-object tracking pipelines. It consists of an index-merge module that adapts the object-centric slots into detection outputs and an object memory module. Benefited from object-centric learning, we only require sparse detection labels for object localization and feature binding.
  arXiv  Detail & Related papers  (2023-09-01T03:34:12Z)
• 3DRP-Net: 3D Relative Position-aware Network for 3D Visual Grounding [58.924180772480504]
  3D visual grounding aims to localize the target object in a 3D point cloud by a free-form language description. We propose a relation-aware one-stage framework, named 3D Relative Position-aware Network (3-Net)
  arXiv  Detail & Related papers  (2023-07-25T09:33:25Z)
• OGC: Unsupervised 3D Object Segmentation from Rigid Dynamics of Point Clouds [4.709764624933227]
  We propose the first unsupervised method, called OGC, to simultaneously identify multiple 3D objects in a single forward pass. We extensively evaluate our method on five datasets, demonstrating the superior performance for object part instance segmentation.
  arXiv  Detail & Related papers  (2022-10-10T07:01:08Z)
• Self-Supervised Interactive Object Segmentation Through a Singulation-and-Grasping Approach [9.029861710944704]
  We propose a robot learning approach to interact with novel objects and collect each object's training label. The Singulation-and-Grasping (SaG) policy is trained through end-to-end reinforcement learning. Our system achieves 70% singulation success rate in simulated cluttered scenes.
  arXiv  Detail & Related papers  (2022-07-19T15:01:36Z)
• Object-Aware Multi-Branch Relation Networks for Spatio-Temporal Video Grounding [90.12181414070496]
  We propose a novel object-aware multi-branch relation network for object-aware relation discovery. We then propose multi-branch reasoning to capture critical object relationships between the main branch and auxiliary branches.
  arXiv  Detail & Related papers  (2020-08-16T15:39:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.