Following Motion for Sequential Modeling in Video Frame Interpolation

• URL: http://arxiv.org/abs/2608.22861v1
• Date: Mon, 24 Aug 2026 06:43:48 GMT
• Title: Following Motion for Sequential Modeling in Video Frame Interpolation
• Abstract summary: State Space Models (SSMs) have surfaced as a promising architecture in Video Frame Interpolation (VFI)<n>We propose Motion-Guided Mamba for Video Frame Interpolation (MGMVFI), an adaptation of the selective state space model tailored explicitly for VFI.
• Score: 22.71397762551783
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: State Space Models (SSMs) have surfaced as a promising architecture in Video Frame Interpolation (VFI), as they can capture long-range dependencies with linear computational complexity. However, their predefined scanning order limits their effectiveness in modeling the dynamic motion trajectories inherent in VFI problems. To tackle this challenge, we propose Motion-Guided Mamba for Video Frame Interpolation (MGMVFI), an adaptation of the selective state space model tailored explicitly for VFI. MGMVFI introduces Motion-Guided Serialization (MGS), which leverages optical flow to define a motion-adaptive 1D input order for the SSM. This aligns the causal state updates with semantically related tokens, enabling motion-consistent feature propagation, particularly for large and dynamic motions. Additionally, to mitigate the unreliable feature representations caused by inaccurate optical flow estimates, we introduce contextual synthesis that utilizes the surrounding spatial context for robust inter-frame feature synthesis. These components are seamlessly integrated within our tailored Mamba architecture, which also employs a lightweight refinement block to enhance local detail reconstruction at a reduced computational cost. Extensive experiments on standard VFI benchmarks demonstrate that MGMVFI achievesstate-of-the-artperformance,particularly on complex and dynamic motions, thereby establishing a new direction for sequence modeling in video interpolation.

Related papers

• MASS: Motion-Aligned Selective Scan for Refinement in Flow-Based Video Frame Interpolation [18.165045289069703]
  Motion-Aligned Selective Scan (MASS) is a novel framework that reformulates feature scanning from static spatial grids to dynamic motion trajectories.<n>MASS builds a feature sequence along each pixel's flow-guided trajectory and aggregates it with an SSM.
  arXiv  Detail & Related papers  (2026-06-26T04:56:31Z)
• A Renaissance of Explicit Motion Information Mining from Transformers for Action Recognition [87.12969639957441]
  Action recognition has been dominated by transformer-based methods, thanks to their contextual aggregation capacities.<n>We propose to integrate those effective motion modeling properties into the existing transformer in a unified and neat way.<n>Our method performs better than existing state-of-the-art approaches, especially on motion-sensitive datasets.
  arXiv  Detail & Related papers  (2025-10-21T15:01:48Z)
• Motion-Aware Concept Alignment for Consistent Video Editing [57.08108545219043]
  We introduce MoCA-Video (Motion-Aware Concept Alignment in Video), a training-free framework bridging the gap between image-domain semantic mixing and video.<n>Given a generated video and a user-provided reference image, MoCA-Video injects the semantic features of the reference image into a specific object within the video.<n>We evaluate MoCA's performance using the standard SSIM, image-level LPIPS, temporal LPIPS, and introduce a novel metric CASS (Conceptual Alignment Shift Score) to evaluate the consistency and effectiveness of the visual shifts between the source prompt and the modified video frames
  arXiv  Detail & Related papers  (2025-06-01T13:28:04Z)
• Motion-Aware Generative Frame Interpolation [23.380470636851022]
  Flow-based frame methods ensure motion stability through estimated intermediate flow but often introduce severe artifacts in complex motion regions.<n>Recent generative approaches, boosted by large-scale pre-trained video generation models, show promise in handling intricate scenes.<n>We propose Motion-aware Generative frame (MoG) that synergizes intermediate flow guidance with generative capacities to enhance fidelity.
  arXiv  Detail & Related papers  (2025-01-07T11:03:43Z)
• Generalizable Implicit Motion Modeling for Video Frame Interpolation [51.966062283735596]
  Motion is critical in flow-based Video Frame Interpolation (VFI)<n>We introduce General Implicit Motion Modeling (IMM), a novel and effective approach to motion modeling VFI.<n>Our GIMM can be easily integrated with existing flow-based VFI works by supplying accurately modeled motion.
  arXiv  Detail & Related papers  (2024-07-11T17:13:15Z)
• Motion-aware Latent Diffusion Models for Video Frame Interpolation [51.78737270917301]
  Motion estimation between neighboring frames plays a crucial role in avoiding motion ambiguity. We propose a novel diffusion framework, motion-aware latent diffusion models (MADiff) Our method achieves state-of-the-art performance significantly outperforming existing approaches.
  arXiv  Detail & Related papers  (2024-04-21T05:09:56Z)
• Spectral Motion Alignment for Video Motion Transfer using Diffusion Models [54.32923808964701]
  Spectral Motion Alignment (SMA) is a framework that refines and aligns motion vectors using Fourier and wavelet transforms.<n> SMA learns motion patterns by incorporating frequency-domain regularization, facilitating the learning of whole-frame global motion dynamics.<n>Extensive experiments demonstrate SMA's efficacy in improving motion transfer while maintaining computational efficiency and compatibility across various video customization frameworks.
  arXiv  Detail & Related papers  (2024-03-22T14:47:18Z)
• Motion-Aware Video Frame Interpolation [49.49668436390514]
  We introduce a Motion-Aware Video Frame Interpolation (MA-VFI) network, which directly estimates intermediate optical flow from consecutive frames. It not only extracts global semantic relationships and spatial details from input frames with different receptive fields, but also effectively reduces the required computational cost and complexity.
  arXiv  Detail & Related papers  (2024-02-05T11:00:14Z)
• Interactive Character Control with Auto-Regressive Motion Diffusion Models [18.727066177880708]
  We propose A-MDM (Auto-regressive Motion Diffusion Model) for real-time motion synthesis. Our conditional diffusion model takes an initial pose as input, and auto-regressively generates successive motion frames conditioned on previous frame. We introduce a suite of techniques for incorporating interactive controls into A-MDM, such as task-oriented sampling, in-painting, and hierarchical reinforcement learning.
  arXiv  Detail & Related papers  (2023-06-01T07:48:34Z)
• JNMR: Joint Non-linear Motion Regression for Video Frame Interpolation [47.123769305867775]
  Video frame (VFI) aims to generate frames by warping learnable motions from the bidirectional historical references. We reformulate VFI as a Joint Non-linear Motion Regression (JNMR) strategy to model the complicated motions of inter-frame. We show that the effectiveness and significant improvement of joint motion regression compared with state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-06-09T02:47:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.