Rebalancing Reference Frame Dominance to Improve Motion in Image-to-Video Models

• URL: http://arxiv.org/abs/2605.19398v2
• Date: Wed, 20 May 2026 05:07:06 GMT
• Title: Rebalancing Reference Frame Dominance to Improve Motion in Image-to-Video Models
• Abstract summary: Image-to-video models often generate videos that remain overly static, compared to text-to-video models.<n>We identify reference-frame dominance as a key mechanism behind motion suppression.<n>We propose DyMoS, a training-free and model-agnostic method that rebalances the attention pathway from generated frames to the reference frame.
• Score: 27.524568995413087
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Image-to-video models often generate videos that remain overly static, compared to text-to-video models. While prior approaches mitigate this issue by weakening or modifying the image-conditioning signal, they often require additional training or sacrifice fidelity to the reference image. In this work, we identify reference-frame dominance as a key mechanism behind motion suppression. We observe that non-reference frames in I2V models allocate excessive self-attention to reference-frame key tokens, causing reference information to be over-propagated across time and suppressing inter-frame dynamics. Based on this finding, we propose DyMoS (Dynamic Motion Slider), a training-free and model-agnostic method that rebalances the attention pathway from generated frames to the reference frame during initial denoising steps. DyMoS leaves both the input image and model weights unchanged and introduces a single scalar parameter for continuous control over motion strength. Experiments across multiple state-of-the-art I2V backbones demonstrate that DyMoS consistently improves motion dynamics while maintaining visual quality and fidelity to the reference image.

Related papers

• MVFusion-GS: Motion-Variance Guided Temporal Attention for High-Quality Dynamic Gaussian Splatting [7.054490512145752]
  3D Gaussian Splatting (3DGS) enables real-time novel view synthesis for static scenes.<n>Existing deformation networks lack explicit motion awareness.<n>We present MVFusion-GS, a method that enhances deformation networks with two complementary motion-aware mechanisms.
  arXiv  Detail & Related papers  (2026-07-02T01:21:11Z)
• Reward Forcing: Efficient Streaming Video Generation with Rewarded Distribution Matching Distillation [69.57572900337176]
  We introduce Reward Forcing, a novel framework for efficient streaming video generation.<n> EMA-Sink tokens capture both long-term context and recent dynamics, preventing initial frame copying.<n>Re-DMD biases the model's output distribution toward high-reward regions by prioritizing samples with greater dynamics rated by a vision-language model.
  arXiv  Detail & Related papers  (2025-12-04T11:12:13Z)
• SteadyDancer: Harmonized and Coherent Human Image Animation with First-Frame Preservation [50.792027578906804]
  We introduce SteadyDancer, an Image-to-Video (R2V) paradigm-based framework that achieves harmonized and coherent animation.<n> Experiments demonstrate that SteadyDancer achieves state-of-the-art performance in both appearance fidelity and motion control.
  arXiv  Detail & Related papers  (2025-11-24T17:15:55Z)
• Real-Time Motion-Controllable Autoregressive Video Diffusion [79.32730467857535]
  We propose AR-Drag, the first RL-enhanced few-step AR video diffusion model for real-time image-to-video generation with diverse motion control.<n>We first fine-tune a base I2V model to support basic motion control, then further improve it via reinforcement with a trajectory-based reward model.<n>Our design preserves the Markov property through a Self-Rollout learning mechanism and accelerates training by selectively denoising steps.
  arXiv  Detail & Related papers  (2025-10-09T12:17:11Z)
• Consistent and Controllable Image Animation with Motion Linear Diffusion Transformers [23.176184261595747]
  We propose MiraMo, a framework designed to enhance efficiency, appearance consistency, and motion smoothness in image animation.<n>Specifically, MiraMo introduces three key elements: (1) A foundational text-to-video architecture replacing vanilla self-attention with efficient linear attention to reduce computational overhead while preserving generation quality; (2) A novel motion residual learning paradigm that focuses on modeling motion dynamics rather than directly predicting frames, improving temporal consistency; and (3) A DCT-based noise refinement strategy during inference to suppress sudden motion artifacts, complemented by a dynamics control module to balance motion smoothness and expressiveness.
  arXiv  Detail & Related papers  (2025-08-10T08:59:32Z)
• Enhancing Motion Dynamics of Image-to-Video Models via Adaptive Low-Pass Guidance [70.12690940725092]
  adaptive low-pass guidance (ALG) is a simple fix to the I2V model sampling procedure to generate more dynamic videos.<n>Under VBench-I2V test suite, ALG achieves an average improvement of 36% in dynamic degree without a significant drop in video quality or image fidelity.
  arXiv  Detail & Related papers  (2025-06-10T05:23:46Z)
• Motion-Aware Concept Alignment for Consistent Video Editing [57.08108545219043]
  We introduce MoCA-Video (Motion-Aware Concept Alignment in Video), a training-free framework bridging the gap between image-domain semantic mixing and video.<n>Given a generated video and a user-provided reference image, MoCA-Video injects the semantic features of the reference image into a specific object within the video.<n>We evaluate MoCA's performance using the standard SSIM, image-level LPIPS, temporal LPIPS, and introduce a novel metric CASS (Conceptual Alignment Shift Score) to evaluate the consistency and effectiveness of the visual shifts between the source prompt and the modified video frames
  arXiv  Detail & Related papers  (2025-06-01T13:28:04Z)
• Extrapolating and Decoupling Image-to-Video Generation Models: Motion Modeling is Easier Than You Think [24.308538128761985]
  Image-to-Video (I2V) generation aims to synthesize a video clip according to a given image and condition (e.g., text)<n>Key challenge of this task lies in simultaneously generating natural motions while preserving the original appearance of the images.<n>We propose a novel Extrapolating and Decoupling framework, which introduces model merging techniques to the I2V domain for the first time.
  arXiv  Detail & Related papers  (2025-03-02T16:06:16Z)
• VideoJAM: Joint Appearance-Motion Representations for Enhanced Motion Generation in Video Models [71.9811050853964]
  VideoJAM is a novel framework that instills an effective motion prior to video generators.<n>VideoJAM achieves state-of-the-art performance in motion coherence.<n>These findings emphasize that appearance and motion can be complementary and, when effectively integrated, enhance both the visual quality and the coherence of video generation.
  arXiv  Detail & Related papers  (2025-02-04T17:07:10Z)
• Boosting Adversarial Transferability using Dynamic Cues [15.194437322391558]
  We introduce spatial (image) and temporal (video) cues within the same source model through task-specific prompts. Our attack results indicate that the attacker does not need specialized architectures. Image models are effective surrogates to optimize an adversarial attack to fool black-box models in a changing environment.
  arXiv  Detail & Related papers  (2023-02-23T18:59:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.