AffordMatcher: Affordance Learning in 3D Scenes from Visual Signifiers

• URL: http://arxiv.org/abs/2603.27970v1
• Date: Mon, 30 Mar 2026 02:47:55 GMT
• Title: AffordMatcher: Affordance Learning in 3D Scenes from Visual Signifiers
• Abstract summary: We introduce AffordBridge, a large-scale dataset with 291,637 functional interaction annotations across 685 high-resolution indoor scenes in the form of point clouds.<n>Our affordance annotations are complemented by RGB images that are linked to the same instances within the scenes.<n>Building upon our dataset, we propose AffordMatcher, an affordance learning method that establishes coherent semantic correspondences between image-based and point cloud-based instances for keypoint matching.
• Score: 16.517711403672998
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Affordance learning is a complex challenge in many applications, where existing approaches primarily focus on the geometric structures, visual knowledge, and affordance labels of objects to determine interactable regions. However, extending this learning capability to a scene is significantly more complicated, as incorporating object- and scene-level semantics is not straightforward. In this work, we introduce AffordBridge, a large-scale dataset with 291,637 functional interaction annotations across 685 high-resolution indoor scenes in the form of point clouds. Our affordance annotations are complemented by RGB images that are linked to the same instances within the scenes. Building upon our dataset, we propose AffordMatcher, an affordance learning method that establishes coherent semantic correspondences between image-based and point cloud-based instances for keypoint matching, enabling a more precise identification of affordance regions based on cues, so-called visual signifiers. Experimental results on our dataset demonstrate the effectiveness of our approach compared to other methods.

Related papers

• Unveiling the Unknown: Open Vocabulary Object Detection with Scene Graphs [16.874298364235326]
  Open-vocabulary object detection seeks to identify novel object categories that were not part of the training data.<n>These methods often overlook structured, image-specific relationships between objects, such as interactions and spatial arrangements.<n>We propose a Scene-guided Modeling detection framework to address this issue.
  arXiv  Detail & Related papers  (2026-06-04T09:23:08Z)
• Weakly-Supervised Learning of Dense Functional Correspondences [23.794395724229762]
  We propose a weakly-supervised learning paradigm to tackle the prediction task.<n>The main insight behind our approach is that we can leverage vision-language models to obtain functional parts.<n>We then integrate this with dense contrastive learning from pixel correspondences to distill both functional and spatial knowledge into a new model.
  arXiv  Detail & Related papers  (2025-09-04T05:39:16Z)
• Selective Contrastive Learning for Weakly Supervised Affordance Grounding [21.34662128701812]
  Weakly supervised affordance grounding seeks to imitate human learning from third-person demonstrations.<n>We introduce selective prototypical and pixel contrastive objectives that adaptively learn affordance-relevant cues at both the part and object levels.
  arXiv  Detail & Related papers  (2025-08-11T11:49:37Z)
• ResVG: Enhancing Relation and Semantic Understanding in Multiple Instances for Visual Grounding [42.10086029931937]
  Visual grounding aims to localize the object referred to in an image based on a natural language query. Existing methods demonstrate a significant performance drop when there are multiple distractions in an image. We propose a novel approach, the Relation and Semantic-sensitive Visual Grounding (ResVG) model, to address this issue.
  arXiv  Detail & Related papers  (2024-08-29T07:32:01Z)
• Leveraging Open-Vocabulary Diffusion to Camouflaged Instance Segmentation [59.78520153338878]
  Text-to-image diffusion techniques have shown exceptional capability of producing high-quality images from text descriptions. We propose a method built upon a state-of-the-art diffusion model, empowered by open-vocabulary to learn multi-scale textual-visual features for camouflaged object representations.
  arXiv  Detail & Related papers  (2023-12-29T07:59:07Z)
• Open-Vocabulary Camouflaged Object Segmentation [66.94945066779988]
  We introduce a new task, open-vocabulary camouflaged object segmentation (OVCOS) We construct a large-scale complex scene dataset (textbfOVCamo) containing 11,483 hand-selected images with fine annotations and corresponding object classes. By integrating the guidance of class semantic knowledge and the supplement of visual structure cues from the edge and depth information, the proposed method can efficiently capture camouflaged objects.
  arXiv  Detail & Related papers  (2023-11-19T06:00:39Z)
• Improving Human-Object Interaction Detection via Virtual Image Learning [68.56682347374422]
  Human-Object Interaction (HOI) detection aims to understand the interactions between humans and objects. In this paper, we propose to alleviate the impact of such an unbalanced distribution via Virtual Image Leaning (VIL) A novel label-to-image approach, Multiple Steps Image Creation (MUSIC), is proposed to create a high-quality dataset that has a consistent distribution with real images.
  arXiv  Detail & Related papers  (2023-08-04T10:28:48Z)
• Semantic-guided modeling of spatial relation and object co-occurrence for indoor scene recognition [5.083140094792973]
  SpaCoNet simultaneously models Spatial relation and Co-occurrence of objects guided by semantic segmentation.<n> Experimental results on three widely used scene datasets demonstrate the effectiveness and generality of the proposed method.
  arXiv  Detail & Related papers  (2023-05-22T03:04:22Z)
• Learning-based Relational Object Matching Across Views [63.63338392484501]
  We propose a learning-based approach which combines local keypoints with novel object-level features for matching object detections between RGB images. We train our object-level matching features based on appearance and inter-frame and cross-frame spatial relations between objects in an associative graph neural network.
  arXiv  Detail & Related papers  (2023-05-03T19:36:51Z)
• Self-Supervised Visual Representation Learning with Semantic Grouping [50.14703605659837]
  We tackle the problem of learning visual representations from unlabeled scene-centric data. We propose contrastive learning from data-driven semantic slots, namely SlotCon, for joint semantic grouping and representation learning.
  arXiv  Detail & Related papers  (2022-05-30T17:50:59Z)
• Mining Cross-Image Semantics for Weakly Supervised Semantic Segmentation [128.03739769844736]
  Two neural co-attentions are incorporated into the classifier to capture cross-image semantic similarities and differences. In addition to boosting object pattern learning, the co-attention can leverage context from other related images to improve localization map inference. Our algorithm sets new state-of-the-arts on all these settings, demonstrating well its efficacy and generalizability.
  arXiv  Detail & Related papers  (2020-07-03T21:53:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.