Anchoring and Rescaling Attention for Semantically Coherent Inbetweening

• URL: http://arxiv.org/abs/2603.17651v1
• Date: Wed, 18 Mar 2026 12:11:02 GMT
• Title: Anchoring and Rescaling Attention for Semantically Coherent Inbetweening
• Abstract summary: Generative inbetweening (GI) seeks to synthesize realistic intermediate frames between the first and last stages of a sequence.<n>We give semantic and temporal guidance from the paths and text onto each intermediate frame through Keyframe-anchored Attention Bias.<n>We also better frame consistency with Rescaled Temporal RoPE, which allows self-attention to enforces more faithfully.
• Score: 9.061831090604324
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Generative inbetweening (GI) seeks to synthesize realistic intermediate frames between the first and last keyframes beyond mere interpolation. As sequences become sparser and motions larger, previous GI models struggle with inconsistent frames with unstable pacing and semantic misalignment. Since GI involves fixed endpoints and numerous plausible paths, this task requires additional guidance gained from the keyframes and text to specify the intended path. Thus, we give semantic and temporal guidance from the keyframes and text onto each intermediate frame through Keyframe-anchored Attention Bias. We also better enforce frame consistency with Rescaled Temporal RoPE, which allows self-attention to attend to keyframes more faithfully. TGI-Bench, the first benchmark specifically designed for text-conditioned GI evaluation, enables challenge-targeted evaluation to analyze GI models. Without additional training, our method achieves state-of-the-art frame consistency, semantic fidelity, and pace stability for both short and long sequences across diverse challenges.

Related papers

• Single-Stream Multi-Feature Fusion with Temporal Robustness for Gait Emotion Recognition [2.055138638073669]
  3D skeleton-based gait emotion recognition faces high annotation costs, data scarcity, and poor generalization on heterogeneous data.<n>This paper proposes SV-GCN, a single-stream multi-feature fusion framework with temporal invariance.
  arXiv  Detail & Related papers  (2026-09-10T15:11:51Z)
• Beyond Static Frames: Temporal Aggregate-and-Restore Vision Transformer for Human Pose Estimation [7.63512442757446]
  Vision Transformers (ViTs) have recently achieved state-of-the-art performance in 2D human pose estimation.<n> TAR-ViTPose enhances static ViT representations by aggregating temporal cues across frames in a plug-and-play manner.<n>Our approach outperforms existing state-of-the-art video-based methods, while also achieving a noticeably higher runtime frame rate in real-world applications.
  arXiv  Detail & Related papers  (2026-03-06T05:45:04Z)
• Arbitrary Generative Video Interpolation [27.953958715353608]
  Video frame (VFI) generates intermediate frames from given start and end frames.<n>Existing VFI methods are constrained to synthesize a fixed number of intermediate frames.<n>We present ArbInterp, a novel generative VFI framework that enables efficient synthesis at any timestamp.
  arXiv  Detail & Related papers  (2025-10-01T06:57:10Z)
• Identity-Preserving Text-to-Video Generation Guided by Simple yet Effective Spatial-Temporal Decoupled Representations [131.33758144860988]
  Identity-preserving text-to-video (IPT2V) generation aims to create high-fidelity videos with consistent human identity.<n>Current end-to-end frameworks suffer a critical spatial-temporal trade-off.<n>We propose a simple yet effective spatial-temporal decoupled framework that decomposes representations into spatial features for layouts and temporal features for motion dynamics.
  arXiv  Detail & Related papers  (2025-07-07T06:54:44Z)
• EF-VI: Enhancing End-Frame Injection for Video Inbetweening [52.6883373124261]
  Video inbetweening aims to synthesize intermediate video sequences conditioned on the given start and end frames.<n>We propose a novel video inbetweening framework specific to recent and more powerful transformer-based I2V-DMs, termed EF-VI.<n>It efficiently strengthens the end-frame constraint by utilizing an enhanced injection.
  arXiv  Detail & Related papers  (2025-05-27T13:53:50Z)
• Generative Inbetweening through Frame-wise Conditions-Driven Video Generation [63.43583844248389]
  generative inbetweening aims to generate intermediate frame sequences by utilizing two key frames as input.<n>We propose a Frame-wise Conditions-driven Video Generation (FCVG) method that significantly enhances the temporal stability of interpolated video frames.<n>Our FCVG demonstrates the capability to generate temporally stable videos using both linear and non-linear curves.
  arXiv  Detail & Related papers  (2024-12-16T13:19:41Z)
• Thin-Plate Spline-based Interpolation for Animation Line Inbetweening [54.69811179222127]
  Chamfer Distance (CD) is commonly adopted for evaluating inbetweening performance. We propose a simple yet effective method for animation line inbetweening that adopts thin-plate spline-based transformation. Our method outperforms existing approaches by delivering high-quality results with enhanced fluidity.
  arXiv  Detail & Related papers  (2024-08-17T08:05:31Z)
• RIGID: Recurrent GAN Inversion and Editing of Real Face Videos [73.97520691413006]
  GAN inversion is indispensable for applying the powerful editability of GAN to real images. Existing methods invert video frames individually often leading to undesired inconsistent results over time. We propose a unified recurrent framework, named textbfRecurrent vtextbfIdeo textbfGAN textbfInversion and etextbfDiting (RIGID) Our framework learns the inherent coherence between input frames in an end-to-end manner.
  arXiv  Detail & Related papers  (2023-08-11T12:17:24Z)
• Continuous Intermediate Token Learning with Implicit Motion Manifold for Keyframe Based Motion Interpolation [20.314332409748637]
  We propose a novel framework to formulate latent motion with precision-based constraints. Our proposed method demonstrates both superior datasets accuracy and high visual similarity to ground truth motions.
  arXiv  Detail & Related papers  (2023-03-27T05:53:01Z)
• Embracing Consistency: A One-Stage Approach for Spatio-Temporal Video Grounding [35.73830796500975]
  We present an end-to-end one-stage framework, termed Spatio-Temporal Consistency-Aware Transformer (STCAT) To generate the above template under sufficient video- perception, an encoder-decoder architecture is proposed for effective global context modeling. Our method outperforms previous state-of-the-arts with clear margins on two challenging video benchmarks.
  arXiv  Detail & Related papers  (2022-09-27T11:13:04Z)
• Temporal Feature Alignment and Mutual Information Maximization for Video-Based Human Pose Estimation [38.571715193347366]
  We present a novel hierarchical alignment framework for multi-frame human pose estimation. We rank No.1 in the Multi-frame Person Pose Estimation Challenge on benchmark dataset PoseTrack 2017, and obtain state-of-the-art performance on benchmarks Sub-JHMDB and Pose-Track 2018.
  arXiv  Detail & Related papers  (2022-03-29T04:29:16Z)
• Progressively Guide to Attend: An Iterative Alignment Framework for Temporal Sentence Grounding [53.377028000325424]
  We propose an Iterative Alignment Network (IA-Net) for temporal sentence grounding task. We pad multi-modal features with learnable parameters to alleviate the nowhere-to-attend problem of non-matched frame-word pairs. We also devise a calibration module following each attention module to refine the alignment knowledge.
  arXiv  Detail & Related papers  (2021-09-14T02:08:23Z)
• TimeLens: Event-based Video Frame Interpolation [54.28139783383213]
  We introduce Time Lens, a novel indicates equal contribution method that leverages the advantages of both synthesis-based and flow-based approaches. We show an up to 5.21 dB improvement in terms of PSNR over state-of-the-art frame-based and event-based methods.
  arXiv  Detail & Related papers  (2021-06-14T10:33:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.