A Weakly Supervised Amodal Segmenter with Boundary Uncertainty Estimation

• URL: http://arxiv.org/abs/2108.09897v1
• Date: Mon, 23 Aug 2021 02:27:29 GMT
• Title: A Weakly Supervised Amodal Segmenter with Boundary Uncertainty Estimation
• Authors: Khoi Nguyen, Sinisa Todorovic
• Abstract summary: This paper addresses weakly supervised amodal instance segmentation. The goal is to segment both visible and occluded (amodal) object parts, while training provides only ground-truth visible (modal) segmentations.
• Score: 35.103437828235826
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper addresses weakly supervised amodal instance segmentation, where the goal is to segment both visible and occluded (amodal) object parts, while training provides only ground-truth visible (modal) segmentations. Following prior work, we use data manipulation to generate occlusions in training images and thus train a segmenter to predict amodal segmentations of the manipulated data. The resulting predictions on training images are taken as the pseudo-ground truth for the standard training of Mask-RCNN, which we use for amodal instance segmentation of test images. For generating the pseudo-ground truth, we specify a new Amodal Segmenter based on Boundary Uncertainty estimation (ASBU) and make two contributions. First, while prior work uses the occluder's mask, our ASBU uses the occlusion boundary as input. Second, ASBU estimates an uncertainty map of the prediction. The estimated uncertainty regularizes learning such that lower segmentation loss is incurred on regions with high uncertainty. ASBU achieves significant performance improvement relative to the state of the art on the COCOA and KINS datasets in three tasks: amodal instance segmentation, amodal completion, and ordering recovery.

Related papers

• Adapting Segment Anything Model for Unseen Object Instance Segmentation [70.60171342436092]
  Unseen Object Instance (UOIS) is crucial for autonomous robots operating in unstructured environments. We propose UOIS-SAM, a data-efficient solution for the UOIS task. UOIS-SAM integrates two key components: (i) a Heatmap-based Prompt Generator (HPG) to generate class-agnostic point prompts with precise foreground prediction, and (ii) a Hierarchical Discrimination Network (HDNet) that adapts SAM's mask decoder.
  arXiv  Detail & Related papers  (2024-09-23T19:05:50Z)
• From Generalization to Precision: Exploring SAM for Tool Segmentation in Surgical Environments [7.01085327371458]
  We argue that Segment Anything Model drastically over-segment images with high corruption levels, resulting in degraded performance. We employ the ground-truth tool mask to analyze the results of SAM when the best single mask is selected as prediction. We analyze the Endovis18 and Endovis17 instrument segmentation datasets using synthetic corruptions of various strengths and an In-House dataset featuring counterfactually created real-world corruptions.
  arXiv  Detail & Related papers  (2024-02-28T01:33:49Z)
• Amodal Ground Truth and Completion in the Wild [84.54972153436466]
  We use 3D data to establish an automatic pipeline to determine authentic ground truth amodal masks for partially occluded objects in real images. This pipeline is used to construct an amodal completion evaluation benchmark, MP3D-Amodal, consisting of a variety of object categories and labels.
  arXiv  Detail & Related papers  (2023-12-28T18:59:41Z)
• Coarse-to-Fine Amodal Segmentation with Shape Prior [52.38348188589834]
  Amodal object segmentation is a challenging task that involves segmenting both visible and occluded parts of an object. We propose a novel approach called Coarse-to-Fine: C2F-Seg, that addresses this problem by progressively modeling the amodal segmentation.
  arXiv  Detail & Related papers  (2023-08-31T15:56:29Z)
• Amodal Intra-class Instance Segmentation: Synthetic Datasets and Benchmark [17.6780586288079]
  This paper introduces two new amodal datasets for image amodal completion tasks. We also present a point-supervised scheme with layer priors for amodal instance segmentation. Experiments show that our weakly supervised approach outperforms the SOTA fully supervised methods.
  arXiv  Detail & Related papers  (2023-03-12T07:28:36Z)
• Modeling Multimodal Aleatoric Uncertainty in Segmentation with Mixture of Stochastic Expert [24.216869988183092]
  We focus on capturing the data-inherent uncertainty (aka aleatoric uncertainty) in segmentation, typically when ambiguities exist in input images. We propose a novel mixture of experts (MoSE) model, where each expert network estimates a distinct mode of aleatoric uncertainty. We develop a Wasserstein-like loss that directly minimizes the distribution distance between the MoSE and ground truth annotations.
  arXiv  Detail & Related papers  (2022-12-14T16:48:21Z)
• Amodal Segmentation Based on Visible Region Segmentation and Shape Prior [43.40655235118393]
  We propose a framework to mimic the behavior of human and solve the ambiguity in the learning. Our model infers the amodal mask by concentrating on the visible region and utilizing the shape prior in the memory. Experiments show that our proposed model outperforms existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-12-10T11:39:09Z)
• The Devil is in the Boundary: Exploiting Boundary Representation for Basis-based Instance Segmentation [85.153426159438]
  We propose Basis based Instance(B2Inst) to learn a global boundary representation that can complement existing global-mask-based methods. Our B2Inst leads to consistent improvements and accurately parses out the instance boundaries in a scene.
  arXiv  Detail & Related papers  (2020-11-26T11:26:06Z)
• Amodal Segmentation through Out-of-Task and Out-of-Distribution Generalization with a Bayesian Model [19.235173141731885]
  Amodal completion is a visual task that humans perform easily but which is difficult for computer vision algorithms. We formulate amodal segmentation as an out-of-task and out-of-distribution generalization problem. Our algorithm outperforms alternative methods that use the same supervision by a large margin.
  arXiv  Detail & Related papers  (2020-10-25T18:01:26Z)
• An Information Bottleneck Approach for Controlling Conciseness in Rationale Extraction [84.49035467829819]
  We show that it is possible to better manage this trade-off by optimizing a bound on the Information Bottleneck (IB) objective. Our fully unsupervised approach jointly learns an explainer that predicts sparse binary masks over sentences, and an end-task predictor that considers only the extracted rationale.
  arXiv  Detail & Related papers  (2020-05-01T23:26:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.