From Synchrony to Sequence: Exo-to-Ego Generation via Interpolation

• URL: http://arxiv.org/abs/2604.13793v1
• Date: Wed, 15 Apr 2026 12:32:25 GMT
• Title: From Synchrony to Sequence: Exo-to-Ego Generation via Interpolation
• Abstract summary: Exo-Seqto-Ego video generation aims to synthesize a first-person view from a synchronized third-person and corresponding camera poses.<n> synchronized exo-ego data inherently introduces substantial-temporal discontinuities, violating the smooth-motion assumptions of standard video generation benchmarks.<n>We propose Syn2Ex-Forcing, a sequential sequence model that interpolates between the source and target videos to form a single continuous signal.
• Score: 54.68405211129937
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Exo-to-Ego video generation aims to synthesize a first-person video from a synchronized third-person view and corresponding camera poses. While paired supervision is available, synchronized exo-ego data inherently introduces substantial spatio-temporal and geometric discontinuities, violating the smooth-motion assumptions of standard video generation benchmarks. We identify this synchronization-induced jump as the central challenge and propose Syn2Seq-Forcing, a sequential formulation that interpolates between the source and target videos to form a single continuous signal. By reframing Exo2Ego as sequential signal modeling rather than a conventional condition-output task, our approach enables diffusion-based sequence models, e.g. Diffusion Forcing Transformers (DFoT), to capture coherent transitions across frames more effectively. Empirically, we show that interpolating only the videos, without performing pose interpolation already produces significant improvements, emphasizing that the dominant difficulty arises from spatio-temporal discontinuities. Beyond immediate performance gains, this formulation establishes a general and flexible framework capable of unifying both Exo2Ego and Ego2Exo generation within a single continuous sequence model, providing a principled foundation for future research in cross-view video synthesis.

Related papers

• Flex-Forcing: Towards a Unified Autoregressive and Bidirectional Video Diffusion Model [88.42070387375679]
  We introduce Flex-Forcing, a unified training and inference framework that enables a video diffusion model to seamlessly operate under both bidirectional and autoregressive generation regimes.<n>Experiments on multiple video generation benchmarks demonstrate that Flex-Forcing achieves consistently better video quality, long-video stability than strong baselines with a rigid inference schedule.
  arXiv  Detail & Related papers  (2026-07-03T17:34:14Z)
• Can Video Diffusion Models Predict Past Frames? Bidirectional Cycle Consistency for Reversible Interpolation [25.677744104220853]
  Video frame aims to synthesize realistic intermediate frames between given endpoints while adhering to specific motion semantics.<n>We propose a novel bidirectional framework that enforces symmetry between forward and backward generation trajectories.<n>Our method achieves state-of-the-art performance in imaging quality, motion smoothness, and dynamic control on both 37-frame and 73-frame tasks.
  arXiv  Detail & Related papers  (2026-04-02T06:58:46Z)
• DCDM: Divide-and-Conquer Diffusion Models for Consistency-Preserving Video Generation [77.89090846233906]
  We propose a system-level framework, termed the Divide-and-Conquer Diffusion Model (DCDM)<n>DCDM decomposes video consistency modeling into three dedicated components while sharing a unified video generation backbone.<n>We validate our framework on the test set of the CVM Competition at AAAI'26, and the results demonstrate that the proposed strategies effectively address these challenges.
  arXiv  Detail & Related papers  (2026-02-14T07:02:36Z)
• From Frames to Sequences: Temporally Consistent Human-Centric Dense Prediction [22.291273919939957]
  We develop a scalable synthetic data pipeline that generates human frames and motion-aligned sequences with pixel-accurate depth, normals, and masks.<n>We train a unified ViT-based dense predictor that injects an explicit geometric human prior via CSE embeddings.<n>Our two-stage training strategy, combining static pretraining with dynamic sequence supervision, enables the model first to acquire robust spatial representations and then refine temporal consistency across motion-aligned sequences.
  arXiv  Detail & Related papers  (2026-02-02T05:28:58Z)
• Harmony: Harmonizing Audio and Video Generation through Cross-Task Synergy [39.04292189640444]
  Harmony is a novel framework that mechanistically enforces audio-visual synchronization.<n>It establishes a new state-of-the-art, significantly outperforming existing methods in both generation fidelity and, critically, in achieving fine-grained audio-visual synchronization.
  arXiv  Detail & Related papers  (2025-11-26T16:53:05Z)
• SViMo: Synchronized Diffusion for Video and Motion Generation in Hand-object Interaction Scenarios [48.09735396455107]
  Hand-Object Interaction (HOI) generation has significant application potential.<n>Current 3D HOI motion generation approaches heavily rely on predefined 3D object models and lab-captured motion data.<n>We propose a novel framework that combines visual priors and dynamic constraints within a synchronized diffusion process to generate the HOI video and motion simultaneously.
  arXiv  Detail & Related papers  (2025-06-03T05:04:29Z)
• AsynFusion: Towards Asynchronous Latent Consistency Models for Decoupled Whole-Body Audio-Driven Avatars [71.90109867684025]
  Whole-body audio-driven avatar pose and expression generation is a critical task for creating lifelike digital humans.<n>We propose AsynFusion, a novel framework that leverages diffusion transformers to achieve cohesive expression and gesture synthesis.<n>AsynFusion achieves state-of-the-art performance in generating real-time, synchronized whole-body animations.
  arXiv  Detail & Related papers  (2025-05-21T03:28:53Z)
• Tuning-Free Multi-Event Long Video Generation via Synchronized Coupled Sampling [81.37449968164692]
  We propose Synchronized Coupled Sampling (SynCoS), a novel inference framework that synchronizes denoising paths across the entire video.<n>Our approach combines two complementary sampling strategies, which ensure seamless local transitions and enforce global coherence.<n>Extensive experiments show that SynCoS significantly improves multi-event long video generation, achieving smoother transitions and superior long-range coherence.
  arXiv  Detail & Related papers  (2025-03-11T16:43:45Z)
• AR-Diffusion: Asynchronous Video Generation with Auto-Regressive Diffusion [19.98565541640125]
  We introduce Auto-Regressive Diffusion (AR-Diffusion), a novel model that combines the strengths of auto-regressive and diffusion models for flexible video generation.<n>Inspired by auto-regressive generation, we incorporate a non-decreasing constraint on the corruption timesteps of individual frames.<n>This setup, together with temporal causal attention, enables flexible generation of videos with varying lengths while preserving temporal coherence.
  arXiv  Detail & Related papers  (2025-03-10T15:05:59Z)
• Upscale-A-Video: Temporal-Consistent Diffusion Model for Real-World Video Super-Resolution [65.91317390645163]
  Upscale-A-Video is a text-guided latent diffusion framework for video upscaling. It ensures temporal coherence through two key mechanisms: locally, it integrates temporal layers into U-Net and VAE-Decoder, maintaining consistency within short sequences. It also offers greater flexibility by allowing text prompts to guide texture creation and adjustable noise levels to balance restoration and generation.
  arXiv  Detail & Related papers  (2023-12-11T18:54:52Z)
• Modelling Latent Dynamics of StyleGAN using Neural ODEs [52.03496093312985]
  We learn the trajectory of independently inverted latent codes from GANs. The learned continuous trajectory allows us to perform infinite frame and consistent video manipulation. Our method achieves state-of-the-art performance but with much less computation.
  arXiv  Detail & Related papers  (2022-08-23T21:20:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.