Fast Interactive Video Object Segmentation with Graph Neural Networks
- URL: http://arxiv.org/abs/2103.03821v1
- Date: Fri, 5 Mar 2021 17:37:12 GMT
- Title: Fast Interactive Video Object Segmentation with Graph Neural Networks
- Authors: Viktor Varga, Andr\'as L\H{o}rincz
- Abstract summary: We present a graph neural network based approach for tackling the problem of interactive video object segmentation.
Our network operates on superpixel-graphs which allow us to reduce the dimensionality of the problem by several magnitudes.
- Score: 0.0
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Pixelwise annotation of image sequences can be very tedious for humans.
Interactive video object segmentation aims to utilize automatic methods to
speed up the process and reduce the workload of the annotators. Most
contemporary approaches rely on deep convolutional networks to collect and
process information from human annotations throughout the video. However, such
networks contain millions of parameters and need huge amounts of labeled
training data to avoid overfitting. Beyond that, label propagation is usually
executed as a series of frame-by-frame inference steps, which is difficult to
be parallelized and is thus time consuming. In this paper we present a graph
neural network based approach for tackling the problem of interactive video
object segmentation. Our network operates on superpixel-graphs which allow us
to reduce the dimensionality of the problem by several magnitudes. We show,
that our network possessing only a few thousand parameters is able to achieve
state-of-the-art performance, while inference remains fast and can be trained
quickly with very little data.
Related papers
- Dynamic Graph Message Passing Networks for Visual Recognition [112.49513303433606]
Modelling long-range dependencies is critical for scene understanding tasks in computer vision.
A fully-connected graph is beneficial for such modelling, but its computational overhead is prohibitive.
We propose a dynamic graph message passing network, that significantly reduces the computational complexity.
arXiv Detail & Related papers (2022-09-20T14:41:37Z) - GraphVid: It Only Takes a Few Nodes to Understand a Video [0.0]
We propose a concise representation of videos that encode perceptually meaningful features into graphs.
We construct superpixel-based graph representations of videos by considering superpixels as graph nodes.
We leverage Graph Convolutional Networks to process this representation and predict the desired output.
arXiv Detail & Related papers (2022-07-04T12:52:54Z) - End-to-end video instance segmentation via spatial-temporal graph neural
networks [30.748756362692184]
Video instance segmentation is a challenging task that extends image instance segmentation to the video domain.
Existing methods either rely only on single-frame information for the detection and segmentation subproblems or handle tracking as a separate post-processing step.
We propose a novel graph-neural-network (GNN) based method to handle the aforementioned limitation.
arXiv Detail & Related papers (2022-03-07T05:38:08Z) - Event Neural Networks [13.207573300016277]
Event Neural Networks (EvNets) leverage repetition to achieve considerable savings for video inference tasks.
We show that it is possible to transform virtually any conventional neural into an EvNet.
We demonstrate the effectiveness of our method on several state-of-the-art neural networks for both high- and low-level visual processing.
arXiv Detail & Related papers (2021-12-02T00:08:48Z) - RICE: Refining Instance Masks in Cluttered Environments with Graph
Neural Networks [53.15260967235835]
We propose a novel framework that refines the output of such methods by utilizing a graph-based representation of instance masks.
We train deep networks capable of sampling smart perturbations to the segmentations, and a graph neural network, which can encode relations between objects, to evaluate the segmentations.
We demonstrate an application that uses uncertainty estimates generated by our method to guide a manipulator, leading to efficient understanding of cluttered scenes.
arXiv Detail & Related papers (2021-06-29T20:29:29Z) - Attention-guided Temporal Coherent Video Object Matting [78.82835351423383]
We propose a novel deep learning-based object matting method that can achieve temporally coherent matting results.
Its key component is an attention-based temporal aggregation module that maximizes image matting networks' strength.
We show how to effectively solve the trimap generation problem by fine-tuning a state-of-the-art video object segmentation network.
arXiv Detail & Related papers (2021-05-24T17:34:57Z) - Generating Masks from Boxes by Mining Spatio-Temporal Consistencies in
Videos [159.02703673838639]
We introduce a method for generating segmentation masks from per-frame bounding box annotations in videos.
We use our resulting accurate masks for weakly supervised training of video object segmentation (VOS) networks.
The additional data provides substantially better generalization performance leading to state-of-the-art results in both the VOS and more challenging tracking domain.
arXiv Detail & Related papers (2021-01-06T18:56:24Z) - Reducing the Annotation Effort for Video Object Segmentation Datasets [50.893073670389164]
densely labeling every frame with pixel masks does not scale to large datasets.
We use a deep convolutional network to automatically create pseudo-labels on a pixel level from much cheaper bounding box annotations.
We obtain the new TAO-VOS benchmark, which we make publicly available at www.vision.rwth-aachen.de/page/taovos.
arXiv Detail & Related papers (2020-11-02T17:34:45Z) - CRNet: Cross-Reference Networks for Few-Shot Segmentation [59.85183776573642]
Few-shot segmentation aims to learn a segmentation model that can be generalized to novel classes with only a few training images.
With a cross-reference mechanism, our network can better find the co-occurrent objects in the two images.
Experiments on the PASCAL VOC 2012 dataset show that our network achieves state-of-the-art performance.
arXiv Detail & Related papers (2020-03-24T04:55:43Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.