ATCA: an Arc Trajectory Based Model with Curvature Attention for Video Frame Interpolation

• URL: http://arxiv.org/abs/2208.00856v1
• Date: Mon, 1 Aug 2022 13:42:08 GMT
• Title: ATCA: an Arc Trajectory Based Model with Curvature Attention for Video Frame Interpolation
• Authors: Jinfeng Liu and Lingtong Kong and Jie Yang
• Abstract summary: We propose an arc trajectory based model (ATCA) which learns motion prior to only two consecutive frames and also is lightweight. Experiments show that our approach performs better than many SOTA methods with fewer parameters and faster inference speed.
• Score: 10.369068266836154
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Video frame interpolation is a classic and challenging low-level computer vision task. Recently, deep learning based methods have achieved impressive results, and it has been proven that optical flow based methods can synthesize frames with higher quality. However, most flow-based methods assume a line trajectory with a constant velocity between two input frames. Only a little work enforces predictions with curvilinear trajectory, but this requires more than two frames as input to estimate the acceleration, which takes more time and memory to execute. To address this problem, we propose an arc trajectory based model (ATCA), which learns motion prior from only two consecutive frames and also is lightweight. Experiments show that our approach performs better than many SOTA methods with fewer parameters and faster inference speed.

Related papers

• TAPIR: Tracking Any Point with per-frame Initialization and temporal Refinement [64.11385310305612]
  We present a novel model for Tracking Any Point (TAP) that effectively tracks any queried point on any physical surface throughout a video sequence. Our approach employs two stages: (1) a matching stage, which independently locates a suitable candidate point match for the query point on every other frame, and (2) a refinement stage, which updates both the trajectory and query features based on local correlations. The resulting model surpasses all baseline methods by a significant margin on the TAP-Vid benchmark, as demonstrated by an approximate 20% absolute average Jaccard (AJ) improvement on DAVIS.
  arXiv  Detail & Related papers  (2023-06-14T17:07:51Z)
• Streaming Radiance Fields for 3D Video Synthesis [32.856346090347174]
  We present an explicit-grid based method for reconstructing streaming radiance fields for novel view synthesis of real world dynamic scenes. Experiments on challenging video sequences demonstrate that our approach is capable of achieving a training speed of 15 seconds per-frame with competitive rendering quality.
  arXiv  Detail & Related papers  (2022-10-26T16:23:02Z)
• TTVFI: Learning Trajectory-Aware Transformer for Video Frame Interpolation [50.49396123016185]
  Video frame (VFI) aims to synthesize an intermediate frame between two consecutive frames. We propose a novel Trajectory-aware Transformer for Video Frame Interpolation (TTVFI) Our method outperforms other state-of-the-art methods in four widely-used VFI benchmarks.
  arXiv  Detail & Related papers  (2022-07-19T03:37:49Z)
• Enhanced Bi-directional Motion Estimation for Video Frame Interpolation [0.05541644538483946]
  We present a novel yet effective algorithm for motion-based video frame estimation. Our method achieves excellent performance on a broad range of video frame benchmarks.
  arXiv  Detail & Related papers  (2022-06-17T06:08:43Z)
• TimeLens: Event-based Video Frame Interpolation [54.28139783383213]
  We introduce Time Lens, a novel indicates equal contribution method that leverages the advantages of both synthesis-based and flow-based approaches. We show an up to 5.21 dB improvement in terms of PSNR over state-of-the-art frame-based and event-based methods.
  arXiv  Detail & Related papers  (2021-06-14T10:33:47Z)
• FLAVR: Flow-Agnostic Video Representations for Fast Frame Interpolation [97.99012124785177]
  FLAVR is a flexible and efficient architecture that uses 3D space-time convolutions to enable end-to-end learning and inference for video framesupervised. We demonstrate that FLAVR can serve as a useful self- pretext task for action recognition, optical flow estimation, and motion magnification.
  arXiv  Detail & Related papers  (2020-12-15T18:59:30Z)
• All at Once: Temporally Adaptive Multi-Frame Interpolation with Advanced Motion Modeling [52.425236515695914]
  State-of-the-art methods are iterative solutions interpolating one frame at the time. This work introduces a true multi-frame interpolator. It utilizes a pyramidal style network in the temporal domain to complete the multi-frame task in one-shot.
  arXiv  Detail & Related papers  (2020-07-23T02:34:39Z)
• End-to-end Learning for Inter-Vehicle Distance and Relative Velocity Estimation in ADAS with a Monocular Camera [81.66569124029313]
  We propose a camera-based inter-vehicle distance and relative velocity estimation method based on end-to-end training of a deep neural network. The key novelty of our method is the integration of multiple visual clues provided by any two time-consecutive monocular frames. We also propose a vehicle-centric sampling mechanism to alleviate the effect of perspective distortion in the motion field.
  arXiv  Detail & Related papers  (2020-06-07T08:18:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.