DFPN: Deformable Frame Prediction Network

• URL: http://arxiv.org/abs/2105.12794v1
• Date: Wed, 26 May 2021 19:00:19 GMT
• Title: DFPN: Deformable Frame Prediction Network
• Authors: M. Ak{\i}n Y{\i}lmaz, A. Murat Tekalp
• Abstract summary: We propose a deformable frame prediction network (DFPN) for task oriented implicit motion modeling and next frame prediction. Experimental results demonstrate that the proposed DFPN model achieves state of the art results in next frame prediction.
• Score: 10.885590093103344
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Learned frame prediction is a current problem of interest in computer vision and video compression. Although several deep network architectures have been proposed for learned frame prediction, to the best of our knowledge, there is no work based on using deformable convolutions for frame prediction. To this effect, we propose a deformable frame prediction network (DFPN) for task oriented implicit motion modeling and next frame prediction. Experimental results demonstrate that the proposed DFPN model achieves state of the art results in next frame prediction. Our models and results are available at https://github.com/makinyilmaz/DFPN.

Related papers

• Visual Representation Learning with Stochastic Frame Prediction [90.99577838303297]
  This paper revisits the idea of video generation that learns to capture uncertainty in frame prediction. We design a framework that trains a frame prediction model to learn temporal information between frames. We find this architecture allows for combining both objectives in a synergistic and compute-efficient manner.
  arXiv  Detail & Related papers  (2024-06-11T16:05:15Z)
• Advancing Learned Video Compression with In-loop Frame Prediction [177.67218448278143]
  In this paper, we propose an Advanced Learned Video Compression (ALVC) approach with the in-loop frame prediction module. The predicted frame can serve as a better reference than the previously compressed frame, and therefore it benefits the compression performance. The experiments show the state-of-the-art performance of our ALVC approach in learned video compression.
  arXiv  Detail & Related papers  (2022-11-13T19:53:14Z)
• A unified model for continuous conditional video prediction [14.685237010856953]
  Conditional video prediction tasks are normally solved by task-related models. Almost all conditional video prediction models can only achieve discrete prediction. In this paper, we propose a unified model that addresses these two issues at the same time.
  arXiv  Detail & Related papers  (2022-10-11T22:26:59Z)
• VMFormer: End-to-End Video Matting with Transformer [48.97730965527976]
  Video matting aims to predict alpha mattes for each frame from a given input video sequence. Recent solutions to video matting have been dominated by deep convolutional neural networks (CNN) We propose VMFormer: a transformer-based end-to-end method for video matting.
  arXiv  Detail & Related papers  (2022-08-26T17:51:02Z)
• Pyramidal Predictive Network: A Model for Visual-frame Prediction Based on Predictive Coding Theory [1.4610038284393165]
  We propose a novel neural network model for the task of visual-frame prediction. The model is composed of a series of recurrent and convolutional units forming the top-down and bottom-up streams. It learns to predict future frames in a visual sequence, with ConvLSTMs on each layer in the network making local prediction from top to down.
  arXiv  Detail & Related papers  (2022-08-15T06:28:34Z)
• Optimizing Video Prediction via Video Frame Interpolation [53.16726447796844]
  We present a new optimization framework for video prediction via video frame, inspired by photo-realistic results of video framescapes. Our framework is based on optimization with a pretrained differentiable video frame module without the need for a training dataset. Our approach outperforms other video prediction methods that require a large amount of training data or extra semantic information.
  arXiv  Detail & Related papers  (2022-06-27T17:03:46Z)
• VPTR: Efficient Transformers for Video Prediction [14.685237010856953]
  We propose a new Transformer block for video future frames prediction based on an efficient local spatial-temporal separation attention mechanism. Based on this new Transformer block, a fully autoregressive video future frames prediction Transformer is proposed. A non-autoregressive video prediction Transformer is also proposed to increase the inference speed and reduce the accumulated inference errors of its autoregressive counterpart.
  arXiv  Detail & Related papers  (2022-03-29T18:09:09Z)
• Wide and Narrow: Video Prediction from Context and Motion [54.21624227408727]
  We propose a new framework to integrate these complementary attributes to predict complex pixel dynamics through deep networks. We present global context propagation networks that aggregate the non-local neighboring representations to preserve the contextual information over the past frames. We also devise local filter memory networks that generate adaptive filter kernels by storing the motion of moving objects in the memory.
  arXiv  Detail & Related papers  (2021-10-22T04:35:58Z)
• SLPC: a VRNN-based approach for stochastic lidar prediction and completion in autonomous driving [63.87272273293804]
  We propose a new LiDAR prediction framework that is based on generative models namely Variational Recurrent Neural Networks (VRNNs) Our algorithm is able to address the limitations of previous video prediction frameworks when dealing with sparse data by spatially inpainting the depth maps in the upcoming frames. We present a sparse version of VRNNs and an effective self-supervised training method that does not require any labels.
  arXiv  Detail & Related papers  (2021-02-19T11:56:44Z)
• Adaptive Future Frame Prediction with Ensemble Network [15.19884183320726]
  We propose an adaptive update framework for the future frame prediction task. The proposed framework consists of a pre-trained prediction network, a continuous-updating prediction network, and a weight estimation network. Our approach outperforms existing methods especially for dynamically changing scenes.
  arXiv  Detail & Related papers  (2020-11-13T07:08:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.