Moving Object Proposals with Deep Learned Optical Flow for Video Object Segmentation

• URL: http://arxiv.org/abs/2402.08882v1
• Date: Wed, 14 Feb 2024 01:13:55 GMT
• Title: Moving Object Proposals with Deep Learned Optical Flow for Video Object Segmentation
• Authors: Ge Shi and Zhili Yang
• Abstract summary: We propose a state of art architecture of neural networks to get the moving object proposals (MOP) We first train an unsupervised convolutional neural network (UnFlow) to generate optical flow estimation. Then we render the output of optical flow net to a fully convolutional SegNet model.
• Score: 1.551271936792451
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Dynamic scene understanding is one of the most conspicuous field of interest among computer vision community. In order to enhance dynamic scene understanding, pixel-wise segmentation with neural networks is widely accepted. The latest researches on pixel-wise segmentation combined semantic and motion information and produced good performance. In this work, we propose a state of art architecture of neural networks to accurately and efficiently get the moving object proposals (MOP). We first train an unsupervised convolutional neural network (UnFlow) to generate optical flow estimation. Then we render the output of optical flow net to a fully convolutional SegNet model. The main contribution of our work is (1) Fine-tuning the pretrained optical flow model on the brand new DAVIS Dataset; (2) Leveraging fully convolutional neural networks with Encoder-Decoder architecture to segment objects. We developed the codes with TensorFlow, and executed the training and evaluation processes on an AWS EC2 instance.

Related papers

• Hierarchical Graph Pattern Understanding for Zero-Shot VOS [102.21052200245457]
  This paper proposes a new hierarchical graph neural network (GNN) architecture for zero-shot video object segmentation (ZS-VOS) Inspired by the strong ability of GNNs in capturing structural relations, HGPU innovatively leverages motion cues (ie, optical flow) to enhance the high-order representations from the neighbors of target frames.
  arXiv  Detail & Related papers  (2023-12-15T04:13:21Z)
• Efficient Unsupervised Video Object Segmentation Network Based on Motion Guidance [1.5736899098702974]
  This paper proposes a video object segmentation network based on motion guidance. The model comprises a dual-stream network, motion guidance module, and multi-scale progressive fusion module. The experimental results prove the superior performance of the proposed method.
  arXiv  Detail & Related papers  (2022-11-10T06:13:23Z)
• FAMINet: Learning Real-time Semi-supervised Video Object Segmentation with Steepest Optimized Optical Flow [21.45623125216448]
  Semi-supervised video object segmentation (VOS) aims to segment a few moving objects in a video sequence, where these objects are specified by annotation of first frame. The optical flow has been considered in many existing semi-supervised VOS methods to improve the segmentation accuracy. A FAMINet, which consists of a feature extraction network (F), an appearance network (A), a motion network (M), and an integration network (I), is proposed in this study to address the abovementioned problem.
  arXiv  Detail & Related papers  (2021-11-20T07:24:33Z)
• Optical Flow Estimation from a Single Motion-blurred Image [66.2061278123057]
  Motion blur in an image may have practical interests in fundamental computer vision problems. We propose a novel framework to estimate optical flow from a single motion-blurred image in an end-to-end manner.
  arXiv  Detail & Related papers  (2021-03-04T12:45:18Z)
• Feature Flow: In-network Feature Flow Estimation for Video Object Detection [56.80974623192569]
  Optical flow is widely used in computer vision tasks to provide pixel-level motion information. A common approach is to:forward optical flow to a neural network and fine-tune this network on the task dataset. We propose a novel network (IFF-Net) with an textbfIn-network textbfFeature textbfFlow estimation module for video object detection.
  arXiv  Detail & Related papers  (2020-09-21T07:55:50Z)
• MotionSqueeze: Neural Motion Feature Learning for Video Understanding [46.82376603090792]
  Motion plays a crucial role in understanding videos and most state-of-the-art neural models for video classification incorporate motion information. In this work, we replace external and heavy computation of optical flows with internal and light-weight learning of motion features. We demonstrate that the proposed method provides a significant gain on four standard benchmarks for action recognition with only a small amount of additional cost.
  arXiv  Detail & Related papers  (2020-07-20T08:30:14Z)
• Interactive Video Object Segmentation Using Global and Local Transfer Modules [51.93009196085043]
  We develop a deep neural network, which consists of the annotation network (A-Net) and the transfer network (T-Net) Given user scribbles on a frame, A-Net yields a segmentation result based on the encoder-decoder architecture. We train the entire network in two stages, by emulating user scribbles and employing an auxiliary loss.
  arXiv  Detail & Related papers  (2020-07-16T06:49:07Z)
• Embedded Encoder-Decoder in Convolutional Networks Towards Explainable AI [0.0]
  This paper proposes a new explainable convolutional neural network (XCNN) which represents important and driving visual features of stimuli. The experimental results on the CIFAR-10, Tiny ImageNet, and MNIST datasets showed the success of our algorithm (XCNN) to make CNNs explainable.
  arXiv  Detail & Related papers  (2020-06-19T15:49:39Z)
• Motion-Attentive Transition for Zero-Shot Video Object Segmentation [99.44383412488703]
  We present a Motion-Attentive Transition Network (MATNet) for zero-shot object segmentation. An asymmetric attention block, called Motion-Attentive Transition (MAT), is designed within a two-stream encoder. In this way, the encoder becomes deeply internative, allowing for closely hierarchical interactions between object motion and appearance.
  arXiv  Detail & Related papers  (2020-03-09T16:58:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.