Efficient motion-based metrics for video frame interpolation

• URL: http://arxiv.org/abs/2508.09078v2
• Date: Thu, 18 Sep 2025 15:23:38 GMT
• Title: Efficient motion-based metrics for video frame interpolation
• Authors: Conall Daly, Darren Ramsook, Anil Kokaram,
• Abstract summary: We propose a motion metric based on measuring the divergence of motion fields.<n>We then use our new proposed metrics to evaluate a range of state of the art frame metrics.
• Score: 0.3823356975862005
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Video frame interpolation (VFI) offers a way to generate intermediate frames between consecutive frames of a video sequence. Although the development of advanced frame interpolation algorithms has received increased attention in recent years, assessing the perceptual quality of interpolated content remains an ongoing area of research. In this paper, we investigate simple ways to process motion fields, with the purposes of using them as video quality metric for evaluating frame interpolation algorithms. We evaluate these quality metrics using the BVI-VFI dataset which contains perceptual scores measured for interpolated sequences. From our investigation we propose a motion metric based on measuring the divergence of motion fields. This metric correlates reasonably with these perceptual scores (PLCC=0.51) and is more computationally efficient (x2.7 speedup) compared to FloLPIPS (a well known motion-based metric). We then use our new proposed metrics to evaluate a range of state of the art frame interpolation metrics and find our metrics tend to favour more perceptual pleasing interpolated frames that may not score highly in terms of PSNR or SSIM.

Related papers

• An Efficient Quality Metric for Video Frame Interpolation Based on Motion-Field Divergence [0.3823356975862005]
  Video frame is a fundamental tool for temporal video enhancement, but existing quality metrics struggle to evaluate the impact of artefacts effectively.<n>We present $textPSNR_textDIV$, a novel full-reference quality metric that enhances PSNR through motion divergence weighting.<n>Our approach highlights singularities in motion fields which is then used to weight image errors.
  arXiv  Detail & Related papers  (2025-10-01T18:40:38Z)
• Motion-Aware Video Frame Interpolation [49.49668436390514]
  We introduce a Motion-Aware Video Frame Interpolation (MA-VFI) network, which directly estimates intermediate optical flow from consecutive frames. It not only extracts global semantic relationships and spatial details from input frames with different receptive fields, but also effectively reduces the required computational cost and complexity.
  arXiv  Detail & Related papers  (2024-02-05T11:00:14Z)
• IDO-VFI: Identifying Dynamics via Optical Flow Guidance for Video Frame Interpolation with Events [14.098949778274733]
  Event cameras are ideal for capturing inter-frame dynamics with their extremely high temporal resolution. We propose an event-and-frame-based video frame method named IDO-VFI that assigns varying amounts of computation for different sub-regions. Our proposed method maintains high-quality performance while reducing computation time and computational effort by 10% and 17% respectively on Vimeo90K datasets.
  arXiv  Detail & Related papers  (2023-05-17T13:22:21Z)
• PMI Sampler: Patch Similarity Guided Frame Selection for Aerial Action Recognition [52.78234467516168]
  We introduce the concept of patch mutual information (PMI) score to quantify the motion bias between adjacent frames. We present an adaptive frame selection strategy using shifted leaky ReLu and cumulative distribution function. Our method achieves a relative improvement of 2.2 - 13.8% in top-1 accuracy on UAV-Human, 6.8% on NEC Drone, and 9.0% on Diving48 datasets.
  arXiv  Detail & Related papers  (2023-04-14T00:01:11Z)
• A Unified Framework for Event-based Frame Interpolation with Ad-hoc Deblurring in the Wild [72.0226493284814]
  We propose a unified framework for event-based frame that performs deblurring ad-hoc.<n>Our network consistently outperforms previous state-of-the-art methods on frame, single image deblurring, and the joint task of both.
  arXiv  Detail & Related papers  (2023-01-12T18:19:00Z)
• A Perceptual Quality Metric for Video Frame Interpolation [6.743340926667941]
  As video frame results often unique artifacts, existing quality metrics sometimes are not consistent with human perception when measuring the results. Some recent deep learning-based quality metrics are shown more consistent with human judgments, but their performance on videos is compromised since they do not consider temporal information. Our method learns perceptual features directly from videos instead of individual frames.
  arXiv  Detail & Related papers  (2022-10-04T19:56:10Z)
• Meta-Interpolation: Time-Arbitrary Frame Interpolation via Dual Meta-Learning [65.85319901760478]
  We consider processing different time-steps with adaptively generated convolutional kernels in a unified way with the help of meta-learning. We develop a dual meta-learned frame framework to synthesize intermediate frames with the guidance of context information and optical flow.
  arXiv  Detail & Related papers  (2022-07-27T17:36:23Z)
• Correspondence Matters for Video Referring Expression Comprehension [64.60046797561455]
  Video Referring Expression (REC) aims to localize the referent objects described in the sentence to visual regions in the video frames. Existing methods suffer from two problems: 1) inconsistent localization results across video frames; 2) confusion between the referent and contextual objects. We propose a novel Dual Correspondence Network (dubbed as DCNet) which explicitly enhances the dense associations in both the inter-frame and cross-modal manners.
  arXiv  Detail & Related papers  (2022-07-21T10:31:39Z)
• FloLPIPS: A Bespoke Video Quality Metric for Frame Interpoation [4.151439675744056]
  We present a bespoke full reference video quality metric for VFI, FloLPIPS, that builds on the popular perceptual image quality metric, LPIPS. FloLPIPS shows superior correlation performance with subjective ground truth over 12 popular quality assessors.
  arXiv  Detail & Related papers  (2022-07-17T09:07:33Z)
• Asymmetric Bilateral Motion Estimation for Video Frame Interpolation [50.44508853885882]
  We propose a novel video frame algorithm based on asymmetric bilateral motion estimation (ABME) We predict symmetric bilateral motion fields to interpolate an anchor frame. We estimate asymmetric bilateral motions fields from the anchor frame to the input frames. Third, we use the asymmetric fields to warp the input frames backward and reconstruct the intermediate frame.
  arXiv  Detail & Related papers  (2021-08-15T21:11:35Z)
• TimeLens: Event-based Video Frame Interpolation [54.28139783383213]
  We introduce Time Lens, a novel indicates equal contribution method that leverages the advantages of both synthesis-based and flow-based approaches. We show an up to 5.21 dB improvement in terms of PSNR over state-of-the-art frame-based and event-based methods.
  arXiv  Detail & Related papers  (2021-06-14T10:33:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.