FAMINet: Learning Real-time Semi-supervised Video Object Segmentation with Steepest Optimized Optical Flow

• URL: http://arxiv.org/abs/2111.10531v1
• Date: Sat, 20 Nov 2021 07:24:33 GMT
• Title: FAMINet: Learning Real-time Semi-supervised Video Object Segmentation with Steepest Optimized Optical Flow
• Authors: Ziyang Liu, Jingmeng Liu, Weihai Chen, Xingming Wu, and Zhengguo Li
• Abstract summary: Semi-supervised video object segmentation (VOS) aims to segment a few moving objects in a video sequence, where these objects are specified by annotation of first frame. The optical flow has been considered in many existing semi-supervised VOS methods to improve the segmentation accuracy. A FAMINet, which consists of a feature extraction network (F), an appearance network (A), a motion network (M), and an integration network (I), is proposed in this study to address the abovementioned problem.
• Score: 21.45623125216448
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Semi-supervised video object segmentation (VOS) aims to segment a few moving objects in a video sequence, where these objects are specified by annotation of first frame. The optical flow has been considered in many existing semi-supervised VOS methods to improve the segmentation accuracy. However, the optical flow-based semi-supervised VOS methods cannot run in real time due to high complexity of optical flow estimation. A FAMINet, which consists of a feature extraction network (F), an appearance network (A), a motion network (M), and an integration network (I), is proposed in this study to address the abovementioned problem. The appearance network outputs an initial segmentation result based on static appearances of objects. The motion network estimates the optical flow via very few parameters, which are optimized rapidly by an online memorizing algorithm named relaxed steepest descent. The integration network refines the initial segmentation result using the optical flow. Extensive experiments demonstrate that the FAMINet outperforms other state-of-the-art semi-supervised VOS methods on the DAVIS and YouTube-VOS benchmarks, and it achieves a good trade-off between accuracy and efficiency. Our code is available at https://github.com/liuziyang123/FAMINet.

Related papers

• Moving Object Proposals with Deep Learned Optical Flow for Video Object Segmentation [1.551271936792451]
  We propose a state of art architecture of neural networks to get the moving object proposals (MOP) We first train an unsupervised convolutional neural network (UnFlow) to generate optical flow estimation. Then we render the output of optical flow net to a fully convolutional SegNet model.
  arXiv  Detail & Related papers  (2024-02-14T01:13:55Z)
• Hierarchical Graph Pattern Understanding for Zero-Shot VOS [102.21052200245457]
  This paper proposes a new hierarchical graph neural network (GNN) architecture for zero-shot video object segmentation (ZS-VOS) Inspired by the strong ability of GNNs in capturing structural relations, HGPU innovatively leverages motion cues (ie, optical flow) to enhance the high-order representations from the neighbors of target frames.
  arXiv  Detail & Related papers  (2023-12-15T04:13:21Z)
• SimulFlow: Simultaneously Extracting Feature and Identifying Target for Unsupervised Video Object Segmentation [28.19471998380114]
  Unsupervised video object segmentation (UVOS) aims at detecting the primary objects in a given video sequence without any human interposing. Most existing methods rely on two-stream architectures that separately encode the appearance and motion information before fusing them to identify the target and generate object masks. We propose a novel UVOS model called SimulFlow that simultaneously performs feature extraction and target identification.
  arXiv  Detail & Related papers  (2023-11-30T06:44:44Z)
• GAFlow: Incorporating Gaussian Attention into Optical Flow [62.646389181507764]
  We push Gaussian Attention (GA) into the optical flow models to accentuate local properties during representation learning. We introduce a novel Gaussian-Constrained Layer (GCL) which can be easily plugged into existing Transformer blocks. For reliable motion analysis, we provide a new Gaussian-Guided Attention Module (GGAM)
  arXiv  Detail & Related papers  (2023-09-28T07:46:01Z)
• Co-attention Propagation Network for Zero-Shot Video Object Segmentation [91.71692262860323]
  Zero-shot object segmentation (ZS-VOS) aims to segment objects in a video sequence without prior knowledge of these objects. Existing ZS-VOS methods often struggle to distinguish between foreground and background or to keep track of the foreground in complex scenarios. We propose an encoder-decoder-based hierarchical co-attention propagation network (HCPN) capable of tracking and segmenting objects.
  arXiv  Detail & Related papers  (2023-04-08T04:45:48Z)
• Efficient Unsupervised Video Object Segmentation Network Based on Motion Guidance [1.5736899098702974]
  This paper proposes a video object segmentation network based on motion guidance. The model comprises a dual-stream network, motion guidance module, and multi-scale progressive fusion module. The experimental results prove the superior performance of the proposed method.
  arXiv  Detail & Related papers  (2022-11-10T06:13:23Z)
• Motion-inductive Self-supervised Object Discovery in Videos [99.35664705038728]
  We propose a model for processing consecutive RGB frames, and infer the optical flow between any pair of frames using a layered representation. We demonstrate superior performance over previous state-of-the-art methods on three public video segmentation datasets.
  arXiv  Detail & Related papers  (2022-10-01T08:38:28Z)
• Hierarchical Feature Alignment Network for Unsupervised Video Object Segmentation [99.70336991366403]
  We propose a concise, practical, and efficient architecture for appearance and motion feature alignment. The proposed HFAN reaches a new state-of-the-art performance on DAVIS-16, achieving 88.7 $mathcalJ&mathcalF$ Mean, i.e., a relative improvement of 3.5% over the best published result.
  arXiv  Detail & Related papers  (2022-07-18T10:10:14Z)
• EM-driven unsupervised learning for efficient motion segmentation [3.5232234532568376]
  This paper presents a CNN-based fully unsupervised method for motion segmentation from optical flow. We use the Expectation-Maximization (EM) framework to leverage the loss function and the training procedure of our motion segmentation neural network. Our method outperforms comparable unsupervised methods and is very efficient.
  arXiv  Detail & Related papers  (2022-01-06T14:35:45Z)
• Learning Dynamic Network Using a Reuse Gate Function in Semi-supervised Video Object Segmentation [27.559093073097483]
  Current approaches for Semi-supervised Video Object (Semi-VOS) propagates information from previous frames to generate segmentation mask for the current frame. We exploit this observation by using temporal information to quickly identify frames with minimal change. We propose a novel dynamic network that estimates change across frames and decides which path -- computing a full network or reusing previous frame's feature -- to choose.
  arXiv  Detail & Related papers  (2020-12-21T19:40:17Z)
• Feature Flow: In-network Feature Flow Estimation for Video Object Detection [56.80974623192569]
  Optical flow is widely used in computer vision tasks to provide pixel-level motion information. A common approach is to:forward optical flow to a neural network and fine-tune this network on the task dataset. We propose a novel network (IFF-Net) with an textbfIn-network textbfFeature textbfFlow estimation module for video object detection.
  arXiv  Detail & Related papers  (2020-09-21T07:55:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.