SODAR: Segmenting Objects by DynamicallyAggregating Neighboring Mask
Representations
- URL: http://arxiv.org/abs/2202.07402v1
- Date: Tue, 15 Feb 2022 13:53:03 GMT
- Title: SODAR: Segmenting Objects by DynamicallyAggregating Neighboring Mask
Representations
- Authors: Tao Wang, Jun Hao Liew, Yu Li, Yunpeng Chen, Jiashi Feng
- Abstract summary: Recent state-of-the-art one-stage instance segmentation model SOLO divides the input image into a grid and directly predicts per grid cell object masks with fully-convolutional networks.
We observe SOLO generates similar masks for an object at nearby grid cells, and these neighboring predictions can complement each other as some may better segment certain object part.
Motivated by the observed gap, we develop a novel learning-based aggregation method that improves upon SOLO by leveraging the rich neighboring information.
- Score: 90.8752454643737
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Recent state-of-the-art one-stage instance segmentation model SOLO divides
the input image into a grid and directly predicts per grid cell object masks
with fully-convolutional networks, yielding comparably good performance as
traditional two-stage Mask R-CNN yet enjoying much simpler architecture and
higher efficiency. We observe SOLO generates similar masks for an object at
nearby grid cells, and these neighboring predictions can complement each other
as some may better segment certain object part, most of which are however
directly discarded by non-maximum-suppression. Motivated by the observed gap,
we develop a novel learning-based aggregation method that improves upon SOLO by
leveraging the rich neighboring information while maintaining the architectural
efficiency. The resulting model is named SODAR. Unlike the original per grid
cell object masks, SODAR is implicitly supervised to learn mask representations
that encode geometric structure of nearby objects and complement adjacent
representations with context. The aggregation method further includes two novel
designs: 1) a mask interpolation mechanism that enables the model to generate
much fewer mask representations by sharing neighboring representations among
nearby grid cells, and thus saves computation and memory; 2) a deformable
neighbour sampling mechanism that allows the model to adaptively adjust
neighbor sampling locations thus gathering mask representations with more
relevant context and achieving higher performance. SODAR significantly improves
the instance segmentation performance, e.g., it outperforms a SOLO model with
ResNet-101 backbone by 2.2 AP on COCO \texttt{test} set, with only about 3\%
additional computation. We further show consistent performance gain with the
SOLOv2 model.
Related papers
- ColorMAE: Exploring data-independent masking strategies in Masked AutoEncoders [53.3185750528969]
Masked AutoEncoders (MAE) have emerged as a robust self-supervised framework.
We introduce a data-independent method, termed ColorMAE, which generates different binary mask patterns by filtering random noise.
We demonstrate our strategy's superiority in downstream tasks compared to random masking.
arXiv Detail & Related papers (2024-07-17T22:04:00Z) - Object Segmentation from Open-Vocabulary Manipulation Instructions Based on Optimal Transport Polygon Matching with Multimodal Foundation Models [0.8749675983608172]
We consider the task of generating segmentation masks for the target object from an object manipulation instruction.
In this study, we propose a novel method that generates segmentation masks from open vocabulary instructions.
arXiv Detail & Related papers (2024-07-01T05:48:48Z) - UGMAE: A Unified Framework for Graph Masked Autoencoders [67.75493040186859]
We propose UGMAE, a unified framework for graph masked autoencoders.
We first develop an adaptive feature mask generator to account for the unique significance of nodes.
We then design a ranking-based structure reconstruction objective joint with feature reconstruction to capture holistic graph information.
arXiv Detail & Related papers (2024-02-12T19:39:26Z) - HAISTA-NET: Human Assisted Instance Segmentation Through Attention [3.073046540587735]
We propose a novel approach to enable more precise predictions and generate higher-quality segmentation masks.
Our human-assisted segmentation model, HAISTA-NET, augments the existing Strong Mask R-CNN network to incorporate human-specified partial boundaries.
We show that HAISTA-NET outperforms state-of-the art methods such as Mask R-CNN, Strong Mask R-CNN, and Mask2Former.
arXiv Detail & Related papers (2023-05-04T18:39:14Z) - Spatiotemporal Graph Neural Network based Mask Reconstruction for Video
Object Segmentation [70.97625552643493]
This paper addresses the task of segmenting class-agnostic objects in semi-supervised setting.
We propose a novel graph neuralS network (TG-Net) which captures the local contexts by utilizing all proposals.
arXiv Detail & Related papers (2020-12-10T07:57:44Z) - LevelSet R-CNN: A Deep Variational Method for Instance Segmentation [79.20048372891935]
Currently, many state of the art models are based on the Mask R-CNN framework.
We propose LevelSet R-CNN, which combines the best of both worlds by obtaining powerful feature representations.
We demonstrate the effectiveness of our approach on COCO and Cityscapes datasets.
arXiv Detail & Related papers (2020-07-30T17:52:18Z) - PointINS: Point-based Instance Segmentation [117.38579097923052]
Mask representation in instance segmentation with Point-of-Interest (PoI) features is challenging because learning a high-dimensional mask feature for each instance requires a heavy computing burden.
We propose an instance-aware convolution, which decomposes this mask representation learning task into two tractable modules.
Along with instance-aware convolution, we propose PointINS, a simple and practical instance segmentation approach.
arXiv Detail & Related papers (2020-03-13T08:24:58Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.