Weakly Supervised Image Segmentation Beyond Tight Bounding Box Annotations

• URL: http://arxiv.org/abs/2301.12053v1
• Date: Sat, 28 Jan 2023 02:11:36 GMT
• Title: Weakly Supervised Image Segmentation Beyond Tight Bounding Box Annotations
• Authors: Juan Wang and Bin Xia
• Abstract summary: This study investigates whether it is possible to maintain good segmentation performance when loose bounding boxes are used as supervision. The proposed polar transformation based MIL formulation works for both tight and loose bounding boxes. The results demonstrate that the proposed approach achieves state-of-the-art performance for bounding boxes at all precision levels.
• Score: 5.000514512377416
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Weakly supervised image segmentation approaches in the literature usually achieve high segmentation performance using tight bounding box supervision and decrease the performance greatly when supervised by loose bounding boxes. However, compared with loose bounding box, it is much more difficult to acquire tight bounding box due to its strict requirements on the precise locations of the four sides of the box. To resolve this issue, this study investigates whether it is possible to maintain good segmentation performance when loose bounding boxes are used as supervision. For this purpose, this work extends our previous parallel transformation based multiple instance learning (MIL) for tight bounding box supervision by integrating an MIL strategy based on polar transformation to assist image segmentation. The proposed polar transformation based MIL formulation works for both tight and loose bounding boxes, in which a positive bag is defined as pixels in a polar line of a bounding box with one endpoint located inside the object enclosed by the box and the other endpoint located at one of the four sides of the box. Moreover, a weighted smooth maximum approximation is introduced to incorporate the observation that pixels closer to the origin of the polar transformation are more likely to belong to the object in the box. The proposed approach was evaluated on two public datasets using dice coefficient when bounding boxes at different precision levels were considered in the experiments. The results demonstrate that the proposed approach achieves state-of-the-art performance for bounding boxes at all precision levels and is robust to mild and moderate errors in the loose bounding box annotations. The codes are available at \url{https://github.com/wangjuan313/wsis-beyond-tightBB}.

Related papers

• Shape-IoU: More Accurate Metric considering Bounding Box Shape and Scale [5.8666339171606445]
  The Shape IoU method can calculate the loss by focusing on the shape and scale of the bounding box itself. Our method can effectively improve detection performance and outperform existing methods, achieving state-of-the-art performance in different detection tasks.
  arXiv  Detail & Related papers  (2023-12-29T16:05:02Z)
• Rigidity-Aware Detection for 6D Object Pose Estimation [60.88857851869196]
  Most recent 6D object pose estimation methods first use object detection to obtain 2D bounding boxes before actually regressing the pose. We propose a rigidity-aware detection method exploiting the fact that, in 6D pose estimation, the target objects are rigid. Key to the success of our approach is a visibility map, which we propose to build using a minimum barrier distance between every pixel in the bounding box and the box boundary.
  arXiv  Detail & Related papers  (2023-03-22T09:02:54Z)
• BoxPolyp:Boost Generalized Polyp Segmentation Using Extra Coarse Bounding Box Annotations [79.17754846553866]
  We propose a boosted BoxPolyp model to make full use of both accurate mask and extra coarse box annotations. In practice, box annotations are applied to alleviate the over-fitting issue of previous polyp segmentation models. Our proposed model outperforms previous state-of-the-art methods by a large margin.
  arXiv  Detail & Related papers  (2022-12-07T07:45:50Z)
• OPA-3D: Occlusion-Aware Pixel-Wise Aggregation for Monocular 3D Object Detection [51.153003057515754]
  OPA-3D is a single-stage, end-to-end, Occlusion-Aware Pixel-Wise Aggregation network. It jointly estimates dense scene depth with depth-bounding box residuals and object bounding boxes. It outperforms state-of-the-art methods on the main Car category.
  arXiv  Detail & Related papers  (2022-11-02T14:19:13Z)
• Polar Transformation Based Multiple Instance Learning Assisting Weakly Supervised Image Segmentation With Loose Bounding Box Annotations [5.000514512377416]
  This study investigates weakly supervised image segmentation using loose bounding box supervision. It presents a multiple instance learning strategy based on polar transformation to assist image segmentation when loose bounding boxes are employed as supervision.
  arXiv  Detail & Related papers  (2022-03-03T00:44:40Z)
• Bounding Box Tightness Prior for Weakly Supervised Image Segmentation [5.517632401040172]
  It proposes generalized multiple instance learning (MIL) and smooth maximum approximation to integrate the bounding box tightness prior to the deep neural network in an end-to-end manner. The proposed approach was evaluated on two pubic medical datasets using Dice coefficient.
  arXiv  Detail & Related papers  (2021-10-03T06:19:20Z)
• Medical image segmentation with imperfect 3D bounding boxes [0.0]
  We focus on 3D medical images with their corresponding 3D bounding boxes which are considered as series of per-slice non-tight 2D bounding boxes. While current weakly-supervised approaches that use 2D bounding boxes as weak labels can be applied to medical image segmentation, we show that their success is limited in cases when the assumption about the tightness of the bounding boxes breaks. We propose a new bounding box correction framework which is trained on a small set of pixel-level annotations to improve the tightness of a larger set of non-tight bounding box annotations.
  arXiv  Detail & Related papers  (2021-08-06T20:51:20Z)
• Look Closer to Segment Better: Boundary Patch Refinement for Instance Segmentation [51.59290734837372]
  We propose a conceptually simple yet effective post-processing refinement framework to improve the boundary quality. The proposed BPR framework yields significant improvements over the Mask R-CNN baseline on Cityscapes benchmark. By applying the BPR framework to the PolyTransform + SegFix baseline, we reached 1st place on the Cityscapes leaderboard.
  arXiv  Detail & Related papers  (2021-04-12T07:10:48Z)
• Dive Deeper Into Box for Object Detection [49.923586776690115]
  We propose a box reorganization method(DDBNet), which can dive deeper into the box for more accurate localization. Experimental results show that our method is effective which leads to state-of-the-art performance for object detection.
  arXiv  Detail & Related papers  (2020-07-15T07:49:05Z)
• DeepStrip: High Resolution Boundary Refinement [60.00241966809684]
  We propose to convert regions of interest into strip images and compute a boundary prediction in the strip domain. To detect the target boundary, we present a framework with two prediction layers. We enforce a matching consistency and C0 continuity regularization to the network to reduce false alarms.
  arXiv  Detail & Related papers  (2020-03-25T22:44:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.