MoCA: Identity-Preserving Text-to-Video Generation via Mixture of Cross Attention

• URL: http://arxiv.org/abs/2508.03034v2
• Date: Wed, 13 Aug 2025 07:25:50 GMT
• Title: MoCA: Identity-Preserving Text-to-Video Generation via Mixture of Cross Attention
• Authors: Qi Xie, Yongjia Ma, Donglin Di, Xuehao Gao, Xun Yang,
• Abstract summary: MoCA is a Video Diffusion Model built on a Diffusion Transformer (DiT) backbone.<n>Our framework improves inter-frame identity consistency by embedding MoCA layers into each DiT block.<n>We also incorporate a Latent Video Perceptual Loss to enhance identity coherence and fine-grained details across video frames.
• Score: 20.494388126869424
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Achieving ID-preserving text-to-video (T2V) generation remains challenging despite recent advances in diffusion-based models. Existing approaches often fail to capture fine-grained facial dynamics or maintain temporal identity coherence. To address these limitations, we propose MoCA, a novel Video Diffusion Model built on a Diffusion Transformer (DiT) backbone, incorporating a Mixture of Cross-Attention mechanism inspired by the Mixture-of-Experts paradigm. Our framework improves inter-frame identity consistency by embedding MoCA layers into each DiT block, where Hierarchical Temporal Pooling captures identity features over varying timescales, and Temporal-Aware Cross-Attention Experts dynamically model spatiotemporal relationships. We further incorporate a Latent Video Perceptual Loss to enhance identity coherence and fine-grained details across video frames. To train this model, we collect CelebIPVid, a dataset of 10,000 high-resolution videos from 1,000 diverse individuals, promoting cross-ethnicity generalization. Extensive experiments on CelebIPVid show that MoCA outperforms existing T2V methods by over 5% across Face similarity.

Related papers

• DreamID-V:Bridging the Image-to-Video Gap for High-Fidelity Face Swapping via Diffusion Transformer [21.788582116033684]
  Video Face Swapping (VFS) requires seamlessly injecting a source identity into a target video.<n>Existing methods struggle to maintain identity similarity and attribute preservation while preserving temporal consistency.<n>We propose a comprehensive framework to seamlessly transfer the superiority of Image Face Swapping to the video domain.
  arXiv  Detail & Related papers  (2026-01-04T08:07:11Z)
• Uniform Discrete Diffusion with Metric Path for Video Generation [103.86033350602908]
  Continuous-space video generation has advanced rapidly, while discrete approaches lag behind due to error accumulation and long-duration inconsistency.<n>We present Uniform generative modeling and present Uniform pAth (URSA), a powerful framework that bridges the gap with continuous approaches for scalable video generation.<n>URSA consistently outperforms existing discrete methods and achieves performance comparable to state-of-the-art continuous diffusion methods.
  arXiv  Detail & Related papers  (2025-10-28T17:59:57Z)
• BindWeave: Subject-Consistent Video Generation via Cross-Modal Integration [56.98981194478512]
  We propose a unified framework that handles a broad range of subject-to-video scenarios.<n>We introduce an MLLM-DiT framework in which a pretrained multimodal large language model performs deep cross-modal reasoning to ground entities.<n>Experiments on the OpenS2V benchmark demonstrate that our method achieves superior performance across subject consistency, naturalness, and text relevance in generated videos.
  arXiv  Detail & Related papers  (2025-10-01T02:41:11Z)
• LongScape: Advancing Long-Horizon Embodied World Models with Context-Aware MoE [16.561410415129778]
  LongScape is a hybrid framework that combines intra-chunk diffusion denoising with inter-chunk autoregressive causal generation.<n>Our core innovation is an action-guided, variable-length chunking mechanism that partitions video based on the semantic context of robotic actions.
  arXiv  Detail & Related papers  (2025-09-26T02:47:05Z)
• LoViC: Efficient Long Video Generation with Context Compression [68.22069741704158]
  We introduce LoViC, a DiT-based framework trained on million-scale open-domain videos.<n>At the core of our approach is FlexFormer, an expressive autoencoder that jointly compresses video and text into unified latent representations.
  arXiv  Detail & Related papers  (2025-07-17T09:46:43Z)
• Long-Term TalkingFace Generation via Motion-Prior Conditional Diffusion Model [64.11605839142348]
  We introduce the textbfMotion-priors textbfConditional textbfDiffusion textbfModel (textbfMCDM), which utilizes both archived and current clip motion priors to enhance motion prediction and ensure temporal consistency.<n>We also release the textbfTalkingFace-Wild dataset, a multilingual collection of over 200 hours of footage across 10 languages.
  arXiv  Detail & Related papers  (2025-02-13T17:50:23Z)
• VividFace: A Diffusion-Based Hybrid Framework for High-Fidelity Video Face Swapping [43.30061680192465]
  We present the first diffusion-based framework specifically designed for video face swapping.<n>Our approach incorporates a specially designed diffusion model coupled with a VidFaceVAE.<n>Our framework achieves superior performance in identity preservation, temporal consistency, and visual quality compared to existing methods.
  arXiv  Detail & Related papers  (2024-12-15T18:58:32Z)
• ACDiT: Interpolating Autoregressive Conditional Modeling and Diffusion Transformer [95.80384464922147]
  ACDiT is a blockwise Conditional Diffusion Transformer.<n>It offers a flexible between token-wise autoregression and full-sequence diffusion.<n>We show that ACDiT performs best among all autoregressive baselines on image and video generation tasks.
  arXiv  Detail & Related papers  (2024-12-10T18:13:20Z)
• HiFiVFS: High Fidelity Video Face Swapping [35.49571526968986]
  Face swapping aims to generate results that combine the identity from the source with attributes from the target.<n>We propose a high fidelity video face swapping framework, which leverages the strong generative capability and temporal prior of Stable Video Diffusion.<n>Our method achieves state-of-the-art (SOTA) in video face swapping, both qualitatively and quantitatively.
  arXiv  Detail & Related papers  (2024-11-27T12:30:24Z)
• Redefining Temporal Modeling in Video Diffusion: The Vectorized Timestep Approach [29.753974393652356]
  We propose a frame-aware video diffusion model(FVDM) Our approach allows each frame to follow an independent noise schedule, enhancing the model's capacity to capture fine-grained temporal dependencies. Our empirical evaluations show that FVDM outperforms state-of-the-art methods in video generation quality, while also excelling in extended tasks.
  arXiv  Detail & Related papers  (2024-10-04T05:47:39Z)
• Vivid-ZOO: Multi-View Video Generation with Diffusion Model [76.96449336578286]
  New challenges lie in the lack of massive captioned multi-view videos and the complexity of modeling such multi-dimensional distribution. We propose a novel diffusion-based pipeline that generates high-quality multi-view videos centered around a dynamic 3D object from text.
  arXiv  Detail & Related papers  (2024-06-12T21:44:04Z)
• S2DM: Sector-Shaped Diffusion Models for Video Generation [2.0270353391739637]
  We propose a novel Sector-Shaped Diffusion Model (S2DM) for video generation. S2DM can generate a group of intrinsically related data sharing the same semantic and intrinsically related features. We show that, without additional training, our model integrated with another temporal conditions generative model can still achieve comparable performance with existing works.
  arXiv  Detail & Related papers  (2024-03-20T08:50:15Z)
• Swap Attention in Spatiotemporal Diffusions for Text-to-Video Generation [55.36617538438858]
  We propose a novel approach that strengthens the interaction between spatial and temporal perceptions. We curate a large-scale and open-source video dataset called HD-VG-130M.
  arXiv  Detail & Related papers  (2023-05-18T11:06:15Z)
• Video Probabilistic Diffusion Models in Projected Latent Space [75.4253202574722]
  We propose a novel generative model for videos, coined projected latent video diffusion models (PVDM) PVDM learns a video distribution in a low-dimensional latent space and thus can be efficiently trained with high-resolution videos under limited resources.
  arXiv  Detail & Related papers  (2023-02-15T14:22:34Z)
• Spatiotemporal Inconsistency Learning for DeepFake Video Detection [51.747219106855624]
  We present a novel temporal modeling paradigm in TIM by exploiting the temporal difference over adjacent frames along with both horizontal and vertical directions. And the ISM simultaneously utilizes the spatial information from SIM and temporal information from TIM to establish a more comprehensive spatial-temporal representation.
  arXiv  Detail & Related papers  (2021-09-04T13:05:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.