Stateless actor-critic for instance segmentation with high-level priors
- URL: http://arxiv.org/abs/2107.02600v2
- Date: Tue, 21 Mar 2023 15:11:44 GMT
- Title: Stateless actor-critic for instance segmentation with high-level priors
- Authors: Paul Hilt, Maedeh Zarvandi, Edgar Kaziakhmedov, Sourabh Bhide, Maria
Leptin, Constantin Pape, Anna Kreshuk
- Abstract summary: Instance segmentation is an important computer vision problem which remains challenging due to deep learning-based methods.
We formulate the instance segmentation problem as graph partitioning and the actor critic predicts the edge weights driven by the rewards, which are based on the conformity of segmented instances to high-level priors on object shape, position or size.
Experiments on toy and real datasets demonstrate that we can achieve excellent performance without any direct supervision based only on a rich set of priors.
- Score: 3.752550648610726
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Instance segmentation is an important computer vision problem which remains
challenging despite impressive recent advances due to deep learning-based
methods. Given sufficient training data, fully supervised methods can yield
excellent performance, but annotation of ground-truth data remains a major
bottleneck, especially for biomedical applications where it has to be performed
by domain experts. The amount of labels required can be drastically reduced by
using rules derived from prior knowledge to guide the segmentation. However,
these rules are in general not differentiable and thus cannot be used with
existing methods. Here, we relax this requirement by using stateless actor
critic reinforcement learning, which enables non-differentiable rewards. We
formulate the instance segmentation problem as graph partitioning and the actor
critic predicts the edge weights driven by the rewards, which are based on the
conformity of segmented instances to high-level priors on object shape,
position or size. The experiments on toy and real datasets demonstrate that we
can achieve excellent performance without any direct supervision based only on
a rich set of priors.
Related papers
- Adaptive End-to-End Metric Learning for Zero-Shot Cross-Domain Slot
Filling [2.6056468338837457]
Slot filling poses a critical challenge to handle a novel domain whose samples are never seen during training.
Most prior works deal with this problem in a two-pass pipeline manner based on metric learning.
We propose a new adaptive end-to-end metric learning scheme for the challenging zero-shot slot filling.
arXiv Detail & Related papers (2023-10-23T19:01:16Z) - Learning Context-aware Classifier for Semantic Segmentation [88.88198210948426]
In this paper, contextual hints are exploited via learning a context-aware classifier.
Our method is model-agnostic and can be easily applied to generic segmentation models.
With only negligible additional parameters and +2% inference time, decent performance gain has been achieved on both small and large models.
arXiv Detail & Related papers (2023-03-21T07:00:35Z) - Points2Polygons: Context-Based Segmentation from Weak Labels Using
Adversarial Networks [0.0]
In applied image segmentation tasks, the ability to provide numerous and precise labels for training is paramount to the accuracy of the model at inference time.
This overhead is often neglected, and recently proposed segmentation architectures rely heavily on the availability and fidelity of ground truth labels to achieve state-of-the-art accuracies.
We introduce Points2Polygons (P2P), a model which makes use of contextual metric learning techniques that directly addresses this problem.
arXiv Detail & Related papers (2021-06-05T05:17:45Z) - A Simple Baseline for Semi-supervised Semantic Segmentation with Strong
Data Augmentation [74.8791451327354]
We propose a simple yet effective semi-supervised learning framework for semantic segmentation.
A set of simple design and training techniques can collectively improve the performance of semi-supervised semantic segmentation significantly.
Our method achieves state-of-the-art results in the semi-supervised settings on the Cityscapes and Pascal VOC datasets.
arXiv Detail & Related papers (2021-04-15T06:01:39Z) - Sparse Object-level Supervision for Instance Segmentation with Pixel
Embeddings [4.038011160363972]
Most state-of-the-art instance segmentation methods have to be trained on densely annotated images.
We propose a proposal-free segmentation approach based on non-spatial embeddings.
We evaluate the proposed method on challenging 2D and 3D segmentation problems in different microscopy modalities.
arXiv Detail & Related papers (2021-03-26T16:36:56Z) - Unsupervised Semantic Segmentation by Contrasting Object Mask Proposals [78.12377360145078]
We introduce a novel two-step framework that adopts a predetermined prior in a contrastive optimization objective to learn pixel embeddings.
This marks a large deviation from existing works that relied on proxy tasks or end-to-end clustering.
In particular, when fine-tuning the learned representations using just 1% of labeled examples on PASCAL, we outperform supervised ImageNet pre-training by 7.1% mIoU.
arXiv Detail & Related papers (2021-02-11T18:54:47Z) - Semi-supervised Active Learning for Instance Segmentation via Scoring
Predictions [25.408505612498423]
We propose a novel and principled semi-supervised active learning framework for instance segmentation.
Specifically, we present an uncertainty sampling strategy named Triplet Scoring Predictions (TSP) to explicitly incorporate samples ranking clues from classes, bounding boxes and masks.
Results on medical images datasets demonstrate that the proposed method results in the embodiment of knowledge from available data in a meaningful way.
arXiv Detail & Related papers (2020-12-09T02:36:52Z) - The Devil is in Classification: A Simple Framework for Long-tail Object
Detection and Instance Segmentation [93.17367076148348]
We investigate performance drop of the state-of-the-art two-stage instance segmentation model Mask R-CNN on the recent long-tail LVIS dataset.
We unveil that a major cause is the inaccurate classification of object proposals.
We propose a simple calibration framework to more effectively alleviate classification head bias with a bi-level class balanced sampling approach.
arXiv Detail & Related papers (2020-07-23T12:49:07Z) - Naive-Student: Leveraging Semi-Supervised Learning in Video Sequences
for Urban Scene Segmentation [57.68890534164427]
In this work, we ask if we may leverage semi-supervised learning in unlabeled video sequences and extra images to improve the performance on urban scene segmentation.
We simply predict pseudo-labels for the unlabeled data and train subsequent models with both human-annotated and pseudo-labeled data.
Our Naive-Student model, trained with such simple yet effective iterative semi-supervised learning, attains state-of-the-art results at all three Cityscapes benchmarks.
arXiv Detail & Related papers (2020-05-20T18:00:05Z) - Improving Semantic Segmentation via Self-Training [75.07114899941095]
We show that we can obtain state-of-the-art results using a semi-supervised approach, specifically a self-training paradigm.
We first train a teacher model on labeled data, and then generate pseudo labels on a large set of unlabeled data.
Our robust training framework can digest human-annotated and pseudo labels jointly and achieve top performances on Cityscapes, CamVid and KITTI datasets.
arXiv Detail & Related papers (2020-04-30T17:09:17Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.