Occlusion-Aware Instance Segmentation via BiLayer Network Architectures

• URL: http://arxiv.org/abs/2208.04438v1
• Date: Mon, 8 Aug 2022 21:39:26 GMT
• Title: Occlusion-Aware Instance Segmentation via BiLayer Network Architectures
• Authors: Lei Ke, Yu-Wing Tai and Chi-Keung Tang
• Abstract summary: We propose Bilayer Convolutional Network (BCNet), where the top layer detects occluding objects (occluders) and the bottom layer infers partially occluded instances (occludees) We investigate the efficacy of bilayer structure using two popular convolutional network designs, namely, Fully Convolutional Network (FCN) and Graph Convolutional Network (GCN)
• Score: 73.45922226843435
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Segmenting highly-overlapping image objects is challenging, because there is typically no distinction between real object contours and occlusion boundaries on images. Unlike previous instance segmentation methods, we model image formation as a composition of two overlapping layers, and propose Bilayer Convolutional Network (BCNet), where the top layer detects occluding objects (occluders) and the bottom layer infers partially occluded instances (occludees). The explicit modeling of occlusion relationship with bilayer structure naturally decouples the boundaries of both the occluding and occluded instances, and considers the interaction between them during mask regression. We investigate the efficacy of bilayer structure using two popular convolutional network designs, namely, Fully Convolutional Network (FCN) and Graph Convolutional Network (GCN). Further, we formulate bilayer decoupling using the vision transformer (ViT), by representing instances in the image as separate learnable occluder and occludee queries. Large and consistent improvements using one/two-stage and query-based object detectors with various backbones and network layer choices validate the generalization ability of bilayer decoupling, as shown by extensive experiments on image instance segmentation benchmarks (COCO, KINS, COCOA) and video instance segmentation benchmarks (YTVIS, OVIS, BDD100K MOTS), especially for heavy occlusion cases. Code and data are available at https://github.com/lkeab/BCNet.

Related papers

• Adaptive Graph Convolution Module for Salient Object Detection [7.278033100480174]
  We propose an adaptive graph convolution module (AGCM) to deal with complex scenes. Prototype features are extracted from the input image using a learnable region generation layer. The proposed AGCM dramatically improves the SOD performance both quantitatively and quantitatively.
  arXiv  Detail & Related papers  (2023-03-17T07:07:17Z)
• Two-Stream Graph Convolutional Network for Intra-oral Scanner Image Segmentation [133.02190910009384]
  We propose a two-stream graph convolutional network (i.e., TSGCN) to handle inter-view confusion between different raw attributes. Our TSGCN significantly outperforms state-of-the-art methods in 3D tooth (surface) segmentation.
  arXiv  Detail & Related papers  (2022-04-19T10:41:09Z)
• Augmenting Convolutional networks with attention-based aggregation [55.97184767391253]
  We show how to augment any convolutional network with an attention-based global map to achieve non-local reasoning. We plug this learned aggregation layer with a simplistic patch-based convolutional network parametrized by 2 parameters (width and depth) It yields surprisingly competitive trade-offs between accuracy and complexity, in particular in terms of memory consumption.
  arXiv  Detail & Related papers  (2021-12-27T14:05:41Z)
• Recurrence along Depth: Deep Convolutional Neural Networks with Recurrent Layer Aggregation [5.71305698739856]
  This paper introduces a concept of layer aggregation to describe how information from previous layers can be reused to better extract features at the current layer. We propose a very light-weighted module, called recurrent layer aggregation (RLA), by making use of the sequential structure of layers in a deep CNN. Our RLA module is compatible with many mainstream deep CNNs, including ResNets, Xception and MobileNetV2.
  arXiv  Detail & Related papers  (2021-10-22T15:36:33Z)
• Semantic Attention and Scale Complementary Network for Instance Segmentation in Remote Sensing Images [54.08240004593062]
  We propose an end-to-end multi-category instance segmentation model, which consists of a Semantic Attention (SEA) module and a Scale Complementary Mask Branch (SCMB) SEA module contains a simple fully convolutional semantic segmentation branch with extra supervision to strengthen the activation of interest instances on the feature map. SCMB extends the original single mask branch to trident mask branches and introduces complementary mask supervision at different scales.
  arXiv  Detail & Related papers  (2021-07-25T08:53:59Z)
• Contour Proposal Networks for Biomedical Instance Segmentation [0.8602553195689513]
  We present a conceptually simple framework for object instance segmentation called Contour Proposal Network (CPN) CPN detects possibly overlapping objects in an image while simultaneously fitting closed object contours using an interpretable, fixed-sized representation based on Fourier Descriptors. We show CPNs that outperform U-Nets and Mask R-CNNs in instance segmentation accuracy and present variants with execution times suitable for real-time applications.
  arXiv  Detail & Related papers  (2021-04-07T21:00:45Z)
• Deep Occlusion-Aware Instance Segmentation with Overlapping BiLayers [72.38919601150175]
  We propose Bilayer Convolutional Network (BCNet) to segment highly-overlapping objects. BCNet detects the occluding objects (occluder) and the bottom GCN layer infers partially occluded instance (occludee)
  arXiv  Detail & Related papers  (2021-03-23T06:25:42Z)
• Boundary-Aware Segmentation Network for Mobile and Web Applications [60.815545591314915]
  Boundary-Aware Network (BASNet) is integrated with a predict-refine architecture and a hybrid loss for highly accurate image segmentation. BASNet runs at over 70 fps on a single GPU which benefits many potential real applications. Based on BASNet, we further developed two (close to) commercial applications: AR COPY & PASTE, in which BASNet is augmented reality for "COPY" and "PASTING" real-world objects, and OBJECT CUT, which is a web-based tool for automatic object background removal.
  arXiv  Detail & Related papers  (2021-01-12T19:20:26Z)
• Robust Instance Segmentation through Reasoning about Multi-Object Occlusion [9.536947328412198]
  We propose a deep network for multi-object instance segmentation that is robust to occlusion. Our work builds on Compositional Networks, which learn a generative model of neural feature activations to locate occluders. In particular, we obtain feed-forward predictions of the object classes and their instance and occluder segmentations.
  arXiv  Detail & Related papers  (2020-12-03T17:41:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.