DiffMagicFace: Identity Consistent Facial Editing of Real Videos

• URL: http://arxiv.org/abs/2604.13841v1
• Date: Wed, 15 Apr 2026 13:13:04 GMT
• Title: DiffMagicFace: Identity Consistent Facial Editing of Real Videos
• Abstract summary: We introduce DiffMagicFace, a unique video editing framework that integrates two fine-tuned models for text and image control.<n>Our approach does not depend on video datasets but still delivers high-quality results in both consistency and content.
• Score: 16.262125513013746
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Text-conditioned image editing has greatly benefitted from the advancements in Image Diffusion Models. However, extending these techniques to facial video editing introduces challenges in preserving facial identity throughout the source video and ensuring consistency of the edited subject across frames. In this paper, we introduce DiffMagicFace, a unique video editing framework that integrates two fine-tuned models for text and image control. These models operate concurrently during inference to produce video frames that maintain identity features while seamlessly aligning with the editing semantics. To ensure the consistency of the edited videos, we develop a dataset comprising images showcasing various facial perspectives for each edited subject. The creation of a data set is achieved through rendering techniques and the subsequent application of optimization algorithms. Remarkably, our approach does not depend on video datasets but still delivers high-quality results in both consistency and content. The excellent effect holds even for complex tasks like talking head videos and distinguishing closely related categories. The videos edited using our framework exhibit parity with videos that are made using traditional rendering software. Through comparative analysis with current state-of-the-art methods, our framework demonstrates superior performance in both visual appeal and quantitative metrics.

Related papers

• LIVE: Leveraging Image Manipulation Priors for Instruction-based Video Editing [45.1196749483108]
  We propose LIVE, a joint training framework that leverages large-scale, high-quality image editing data alongside video datasets to bolster editing capabilities.<n>We introduce a frame-wise token noise strategy, which treats the latents of specific frames as reasoning tokens, leveraging large pretrained video generative models to create plausible temporal transformations.<n>We curate a comprehensive evaluation benchmark encompassing over 60 challenging tasks that are prevalent in image editing but scarce in existing video datasets.
  arXiv  Detail & Related papers  (2026-04-18T15:09:01Z)
• EditVerse: Unifying Image and Video Editing and Generation with In-Context Learning [58.53074381801114]
  We introduce EditVerse, a unified framework for image and video generation and editing within a single model.<n>By representing all modalities, i.e. text, image, and video, as a unified token sequence, EditVerse leverages self-attention to achieve robust in-context learning.<n>We present EditVerseBench, the first benchmark for instruction-based video editing covering diverse tasks and resolutions.
  arXiv  Detail & Related papers  (2025-09-24T17:59:30Z)
• IP-FaceDiff: Identity-Preserving Facial Video Editing with Diffusion [12.494492016414503]
  Existing models encounter challenges such as poor editing quality, high computational costs and difficulties in preserving facial identity across diverse edits.<n>We propose a novel facial video editing framework that leverages the rich latent space of pre-trained text-to-image (T2I) diffusion models.<n>Our approach significantly reduces editing time by 80%, while maintaining temporal consistency throughout the video sequence.
  arXiv  Detail & Related papers  (2025-01-13T18:08:27Z)
• Portrait Video Editing Empowered by Multimodal Generative Priors [39.747581584889495]
  We introduce PortraitGen, a powerful portrait video editing method that achieves consistent and expressive stylization with multimodal prompts. Our approach incorporates multimodal inputs through knowledge distilled from large-scale 2D generative models. Our system also incorporates expression similarity guidance and a face-aware portrait editing module, effectively mitigating degradation issues associated with iterative dataset updates.
  arXiv  Detail & Related papers  (2024-09-20T15:45:13Z)
• Unified Editing of Panorama, 3D Scenes, and Videos Through Disentangled Self-Attention Injection [60.47731445033151]
  We propose a novel unified editing framework that combines the strengths of both approaches by utilizing only a basic 2D image text-to-image (T2I) diffusion model. Experimental results confirm that our method enables editing across diverse modalities including 3D scenes, videos, and panorama images.
  arXiv  Detail & Related papers  (2024-05-27T04:44:36Z)
• I2VEdit: First-Frame-Guided Video Editing via Image-to-Video Diffusion Models [18.36472998650704]
  We introduce a novel and generic solution that extends the applicability of image editing tools to videos by propagating edits from a single frame to the entire video using a pre-trained image-to-video model. Our method, dubbed I2VEdit, adaptively preserves the visual and motion integrity of the source video depending on the extent of the edits.
  arXiv  Detail & Related papers  (2024-05-26T11:47:40Z)
• MagicProp: Diffusion-based Video Editing via Motion-aware Appearance Propagation [74.32046206403177]
  MagicProp disentangles the video editing process into two stages: appearance editing and motion-aware appearance propagation. In the first stage, MagicProp selects a single frame from the input video and applies image-editing techniques to modify the content and/or style of the frame. In the second stage, MagicProp employs the edited frame as an appearance reference and generates the remaining frames using an autoregressive rendering approach.
  arXiv  Detail & Related papers  (2023-09-02T11:13:29Z)
• Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translation [93.18163456287164]
  This paper proposes a novel text-guided video-to-video translation framework to adapt image models to videos. Our framework achieves global style and local texture temporal consistency at a low cost.
  arXiv  Detail & Related papers  (2023-06-13T17:52:23Z)
• Make-A-Protagonist: Generic Video Editing with An Ensemble of Experts [116.05656635044357]
  We propose a generic video editing framework called Make-A-Protagonist. Specifically, we leverage multiple experts to parse source video, target visual and textual clues, and propose a visual-textual-based video generation model. Results demonstrate the versatile and remarkable editing capabilities of Make-A-Protagonist.
  arXiv  Detail & Related papers  (2023-05-15T17:59:03Z)
• Task-agnostic Temporally Consistent Facial Video Editing [84.62351915301795]
  We propose a task-agnostic, temporally consistent facial video editing framework. Based on a 3D reconstruction model, our framework is designed to handle several editing tasks in a more unified and disentangled manner. Compared with the state-of-the-art facial image editing methods, our framework generates video portraits that are more photo-realistic and temporally smooth.
  arXiv  Detail & Related papers  (2020-07-03T02:49:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.