STRAP: Structured Object Affordance Segmentation with Point Supervision

• URL: http://arxiv.org/abs/2304.08492v1
• Date: Mon, 17 Apr 2023 17:59:49 GMT
• Title: STRAP: Structured Object Affordance Segmentation with Point Supervision
• Authors: Leiyao Cui, Xiaoxue Chen, Hao Zhao, Guyue Zhou, Yixin Zhu
• Abstract summary: We study affordance segmentation with point supervision, wherein the setting inherits an unexplored dual affinity-spatial affinity and label affinity. We devise a dense prediction network that enhances label relations by effectively densifying labels in a new domain. In experiments, we benchmark our method on the challenging CAD120 dataset, showing significant performance gains over prior methods.
• Score: 20.56373848741831
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With significant annotation savings, point supervision has been proven effective for numerous 2D and 3D scene understanding problems. This success is primarily attributed to the structured output space; i.e., samples with high spatial affinity tend to share the same labels. Sharing this spirit, we study affordance segmentation with point supervision, wherein the setting inherits an unexplored dual affinity-spatial affinity and label affinity. By label affinity, we refer to affordance segmentation as a multi-label prediction problem: A plate can be both holdable and containable. By spatial affinity, we refer to a universal prior that nearby pixels with similar visual features should share the same point annotation. To tackle label affinity, we devise a dense prediction network that enhances label relations by effectively densifying labels in a new domain (i.e., label co-occurrence). To address spatial affinity, we exploit a Transformer backbone for global patch interaction and a regularization loss. In experiments, we benchmark our method on the challenging CAD120 dataset, showing significant performance gains over prior methods.

Related papers

• Scribble Hides Class: Promoting Scribble-Based Weakly-Supervised Semantic Segmentation with Its Class Label [16.745019028033518]
  We propose a class-driven scribble promotion network, which utilizes both scribble annotations and pseudo-labels informed by image-level classes and global semantics for supervision. Experiments on the ScribbleSup dataset with different qualities of scribble annotations outperform all the previous methods, demonstrating the superiority and robustness of our method.
  arXiv  Detail & Related papers  (2024-02-27T14:51:56Z)
• Densify Your Labels: Unsupervised Clustering with Bipartite Matching for Weakly Supervised Point Cloud Segmentation [42.144991202299934]
  We propose a weakly supervised semantic segmentation method for point clouds that predicts "per-point" labels from just "whole-scene" annotations. Our core idea is to propagate the scene-level labels to each point in the point cloud by creating pseudo labels in a conservative way. We evaluate our method on ScanNet and S3DIS datasets, outperforming state of the art, and demonstrate that we can achieve results comparable to fully supervised methods.
  arXiv  Detail & Related papers  (2023-12-11T19:18:17Z)
• Semantic Connectivity-Driven Pseudo-labeling for Cross-domain Segmentation [89.41179071022121]
  Self-training is a prevailing approach in cross-domain semantic segmentation. We propose a novel approach called Semantic Connectivity-driven pseudo-labeling. This approach formulates pseudo-labels at the connectivity level and thus can facilitate learning structured and low-noise semantics.
  arXiv  Detail & Related papers  (2023-12-11T12:29:51Z)
• LESS: Label-Efficient Semantic Segmentation for LiDAR Point Clouds [62.49198183539889]
  We propose a label-efficient semantic segmentation pipeline for outdoor scenes with LiDAR point clouds. Our method co-designs an efficient labeling process with semi/weakly supervised learning. Our proposed method is even highly competitive compared to the fully supervised counterpart with 100% labels.
  arXiv  Detail & Related papers  (2022-10-14T19:13:36Z)
• Collaborative Propagation on Multiple Instance Graphs for 3D Instance Segmentation with Single-point Supervision [63.429704654271475]
  We propose a novel weakly supervised method RWSeg that only requires labeling one object with one point. With these sparse weak labels, we introduce a unified framework with two branches to propagate semantic and instance information. Specifically, we propose a Cross-graph Competing Random Walks (CRW) algorithm that encourages competition among different instance graphs.
  arXiv  Detail & Related papers  (2022-08-10T02:14:39Z)
• Graph Attention Transformer Network for Multi-Label Image Classification [50.0297353509294]
  We propose a general framework for multi-label image classification that can effectively mine complex inter-label relationships. Our proposed methods can achieve state-of-the-art performance on three datasets.
  arXiv  Detail & Related papers  (2022-03-08T12:39:05Z)
• Learning Affinity from Attention: End-to-End Weakly-Supervised Semantic Segmentation with Transformers [44.757309147148035]
  We introduce Transformers, which naturally integrate global information, to generate more integral initial pseudo labels for end-to-end WSSS. Motivated by the inherent consistency between the self-attention in Transformers and the semantic affinity, we propose an Affinity from Attention (AFA) module. We also devise a Pixel-Adaptive Refinement module that incorporates low-level image appearance information to refine the pseudo labels.
  arXiv  Detail & Related papers  (2022-03-05T06:07:17Z)
• PointMatch: A Consistency Training Framework for Weakly Supervised Semantic Segmentation of 3D Point Clouds [117.77841399002666]
  We propose a novel framework, PointMatch, that stands on both data and label, by applying consistency regularization to sufficiently probe information from data itself. The proposed PointMatch achieves the state-of-the-art performance under various weakly-supervised schemes on both ScanNet-v2 and S3DIS datasets.
  arXiv  Detail & Related papers  (2022-02-22T07:26:31Z)
• Why Propagate Alone? Parallel Use of Labels and Features on Graphs [42.01561812621306]
  Graph neural networks (GNNs) and label propagation represent two interrelated modeling strategies designed to exploit graph structure in tasks such as node property prediction. We show that a label trick can be reduced to an interpretable, deterministic training objective composed of two factors.
  arXiv  Detail & Related papers  (2021-10-14T07:34:11Z)
• Enhancing Label Correlation Feedback in Multi-Label Text Classification via Multi-Task Learning [6.1538971100140145]
  We introduce a novel approach with multi-task learning to enhance label correlation feedback. We propose two auxiliary label co-occurrence prediction tasks to enhance label correlation learning.
  arXiv  Detail & Related papers  (2021-06-06T12:26:14Z)
• MatchGAN: A Self-Supervised Semi-Supervised Conditional Generative Adversarial Network [51.84251358009803]
  We present a novel self-supervised learning approach for conditional generative adversarial networks (GANs) under a semi-supervised setting. We perform augmentation by randomly sampling sensible labels from the label space of the few labelled examples available. Our method surpasses the baseline with only 20% of the labelled examples used to train the baseline.
  arXiv  Detail & Related papers  (2020-06-11T17:14:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.