S3Editor: A Sparse Semantic-Disentangled Self-Training Framework for Face Video Editing

• URL: http://arxiv.org/abs/2404.08111v1
• Date: Thu, 11 Apr 2024 20:25:26 GMT
• Title: S3Editor: A Sparse Semantic-Disentangled Self-Training Framework for Face Video Editing
• Abstract summary: We introduce S3Editor, a Sparse Semantic-disentangled Self-training framework for face video editing. Firstly, S3Editor adopts a self-training paradigm to enhance the training process through semi-supervision. Secondly, we propose a semantic disentangled architecture with a dynamic routing mechanism that accommodates diverse editing requirements.
• Score: 38.804508101698275
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Face attribute editing plays a pivotal role in various applications. However, existing methods encounter challenges in achieving high-quality results while preserving identity, editing faithfulness, and temporal consistency. These challenges are rooted in issues related to the training pipeline, including limited supervision, architecture design, and optimization strategy. In this work, we introduce S3Editor, a Sparse Semantic-disentangled Self-training framework for face video editing. S3Editor is a generic solution that comprehensively addresses these challenges with three key contributions. Firstly, S3Editor adopts a self-training paradigm to enhance the training process through semi-supervision. Secondly, we propose a semantic disentangled architecture with a dynamic routing mechanism that accommodates diverse editing requirements. Thirdly, we present a structured sparse optimization schema that identifies and deactivates malicious neurons to further disentangle impacts from untarget attributes. S3Editor is model-agnostic and compatible with various editing approaches. Our extensive qualitative and quantitative results affirm that our approach significantly enhances identity preservation, editing fidelity, as well as temporal consistency.

Related papers

• What Makes a 3D Scene Editable? A Factorized Benchmark of Fidelity, Locality, Consistency, and Preservation [0.0]
  We introduce EditBench3D, a representation-agnostic benchmark that treats editing as controlled information replacement.<n>We evaluate eight representative NeRF, 3D Gaussian Splatting, hybrid, and proxy-based editors on 240 scene-edit pairs.
  arXiv  Detail & Related papers  (2026-09-14T01:38:38Z)
• Thinking on Shots: Consistent Multi-Shot Video Editing with Agentic Reasoning [56.17582599802973]
  We introduce the Multi-Instruction Multi-Shot Long-Video Editing (MMLVE) task, which is structured around three core objectives.<n>To tackle these three unique challenges, we introduce an agentic editing framework that leverages the Large Language Models (LLMs) and Vision-Language Models (VLMs)<n>Extensive experiments demonstrate that our MMLVE-Agent outperforms existing closed-source SOTA approaches.
  arXiv  Detail & Related papers  (2026-08-27T08:48:13Z)
• Beyond Voxel 3D Editing: Learning from 3D Masks and Self-Constructed Data [40.44238807341562]
  3D editing requires maintaining semantic consistency by performing localized changes according to prompts.<n>We propose a Beyond Voxel 3D Editing (BVE) framework with a self-constructed large-scale dataset specifically tailored for 3D editing.<n>BVE achieves superior performance in generating high-quality, text-aligned 3D assets, while faithfully retaining the visual characteristics of the original input.
  arXiv  Detail & Related papers  (2026-04-15T10:10:27Z)
• CEI-3D: Collaborative Explicit-Implicit 3D Reconstruction for Realistic and Fine-Grained Object Editing [51.73433734209541]
  Existing 3D editing methods often produce unrealistic and unrefined results due to the deeply integrated nature of their reconstruction networks.<n>This paper introduces CEI-3D, an editing-oriented reconstruction pipeline designed to facilitate realistic and fine-grained editing.<n>Our approach achieves more realistic and fine-grained editing results than the state-of-the-art (SOTA) methods while requiring less editing time.
  arXiv  Detail & Related papers  (2026-03-12T11:15:11Z)
• Optimizing ID Consistency in Multimodal Large Models: Facial Restoration via Alignment, Entanglement, and Disentanglement [54.199726425201895]
  Multimodal editing large models have demonstrated powerful editing capabilities across diverse tasks.<n>Current facial ID preservation methods struggle to achieve consistent restoration of both facial identity and edited element IP.<n>We propose EditedID, an Alignment-Disentanglement-Entanglement framework for robust identity-specific facial restoration.
  arXiv  Detail & Related papers  (2026-02-21T08:24:42Z)
• CoEditor++: Instruction-based Visual Editing via Cognitive Reasoning [98.98349220451216]
  CoEditor++ is a training-free framework that decomposes editing into "what to edit" and "how to edit"<n>We show that CoEditor++ achieves state-of-the-art performance in both general editing and responsible editing tasks.<n>Our findings suggest the potential toward cognitive-centric instruction-based image editing.
  arXiv  Detail & Related papers  (2026-01-31T12:20:46Z)
• Native 3D Editing with Full Attention [47.908091876301796]
  We propose a novel native 3D editing framework that directly manipulates 3D representations in a single, efficient feed-forward pass.<n>This dataset is meticulously curated to ensure that edited objects faithfully adhere to the instructional changes.<n>Our results demonstrate that token concatenation is more parameter-efficient and achieves superior performance.
  arXiv  Detail & Related papers  (2025-11-21T18:59:26Z)
• ConsistEdit: Highly Consistent and Precise Training-free Visual Editing [17.162316662697965]
  We propose ConsistEdit, a novel attention control method specifically tailored for MM-DiT.<n>It incorporates vision-only attention control, mask-guided pre-attention fusion, and differentiated manipulation of the query, key, and value tokens.<n>It achieves state-of-the-art performance across a wide range of image and video editing tasks, including both structure-consistent and structure-inconsistent scenarios.
  arXiv  Detail & Related papers  (2025-10-20T17:59:52Z)
• CoreEditor: Consistent 3D Editing via Correspondence-constrained Diffusion [24.144486805878596]
  CoreEditor is a novel framework for consistent text-to-3D editing.<n>We introduce a correspondence-constrained attention mechanism that enforces precise interactions between pixels.<n>In experiments, CoreEditor produces high-quality, 3D-consistent edits with sharper details.
  arXiv  Detail & Related papers  (2025-08-15T17:13:11Z)
• Image Editing As Programs with Diffusion Models [69.05164729625052]
  We introduce Image Editing As Programs (IEAP), a unified image editing framework built upon the Diffusion Transformer (DiT) architecture.<n>IEAP approaches instructional editing through a reductionist lens, decomposing complex editing instructions into sequences of atomic operations.<n>Our framework delivers superior accuracy and semantic fidelity, particularly for complex, multi-step instructions.
  arXiv  Detail & Related papers  (2025-06-04T16:57:24Z)
• InstructVEdit: A Holistic Approach for Instructional Video Editing [28.13673601495108]
  InstructVEdit is a full-cycle instructional video editing approach that establishes a reliable dataset curation workflow. It incorporates two model architectural improvements to enhance edit quality while preserving temporal consistency. It also proposes an iterative refinement strategy leveraging real-world data to enhance generalization and minimize train-test discrepancies.
  arXiv  Detail & Related papers  (2025-03-22T04:12:20Z)
• SSEditor: Controllable Mask-to-Scene Generation with Diffusion Model [8.664947484778486]
  SSEditor is a controllable Semantic Scene Editor that can generate specified target categories without multiple-step resampling. In the second stage, we introduce a geometric-semantic fusion module that enhance the model's ability to learn geometric and semantic information. experiments on the unseen Occ-3D dataset show that SSEditor is capable of generating novel urban scenes.
  arXiv  Detail & Related papers  (2024-11-19T07:19:05Z)
• Task-Oriented Diffusion Inversion for High-Fidelity Text-based Editing [60.730661748555214]
  We introduce textbfTask-textbfOriented textbfDiffusion textbfInversion (textbfTODInv), a novel framework that inverts and edits real images tailored to specific editing tasks. ToDInv seamlessly integrates inversion and editing through reciprocal optimization, ensuring both high fidelity and precise editability.
  arXiv  Detail & Related papers  (2024-08-23T22:16:34Z)
• View-Consistent 3D Editing with Gaussian Splatting [50.6460814430094]
  View-consistent Editing (VcEdit) is a novel framework that seamlessly incorporates 3DGS into image editing processes. By incorporating consistency modules into an iterative pattern, VcEdit proficiently resolves the issue of multi-view inconsistency.
  arXiv  Detail & Related papers  (2024-03-18T15:22:09Z)
• CNS-Edit: 3D Shape Editing via Coupled Neural Shape Optimization [56.47175002368553]
  This paper introduces a new approach based on a coupled representation and a neural volume optimization to implicitly perform 3D shape editing in latent space. First, we design the coupled neural shape representation for supporting 3D shape editing. Second, we formulate the coupled neural shape optimization procedure to co-optimize the two coupled components in the representation subject to the editing operation.
  arXiv  Detail & Related papers  (2024-02-04T01:52:56Z)
• Plasticine3D: 3D Non-Rigid Editing with Text Guidance by Multi-View Embedding Optimization [21.8454418337306]
  We propose Plasticine3D, a novel text-guided controlled 3D editing pipeline that can perform 3D non-rigid editing. Our work divides the editing process into a geometry editing stage and a texture editing stage to achieve separate control of structure and appearance. For the purpose of fine-grained control, we propose Embedding-Fusion (EF) to blend the original characteristics with the editing objectives in the embedding space.
  arXiv  Detail & Related papers  (2023-12-15T09:01:54Z)
• Editing 3D Scenes via Text Prompts without Retraining [80.57814031701744]
  DN2N is a text-driven editing method that allows for the direct acquisition of a NeRF model with universal editing capabilities. Our method employs off-the-shelf text-based editing models of 2D images to modify the 3D scene images. Our method achieves multiple editing types, including but not limited to appearance editing, weather transition, material changing, and style transfer.
  arXiv  Detail & Related papers  (2023-09-10T02:31:50Z)
• SINE: Semantic-driven Image-based NeRF Editing with Prior-guided Editing Field [37.8162035179377]
  We present a novel semantic-driven NeRF editing approach, which enables users to edit a neural radiance field with a single image. To achieve this goal, we propose a prior-guided editing field to encode fine-grained geometric and texture editing in 3D space. Our method achieves photo-realistic 3D editing using only a single edited image, pushing the bound of semantic-driven editing in 3D real-world scenes.
  arXiv  Detail & Related papers  (2023-03-23T13:58:11Z)
• IDE-3D: Interactive Disentangled Editing for High-Resolution 3D-aware Portrait Synthesis [38.517819699560945]
  Our system consists of three major components: (1) a 3D-semantics-aware generative model that produces view-consistent, disentangled face images and semantic masks; (2) a hybrid GAN inversion approach that initializes the latent codes from the semantic and texture encoder, and further optimized them for faithful reconstruction; and (3) a canonical editor that enables efficient manipulation of semantic masks in canonical view and product high-quality editing results.
  arXiv  Detail & Related papers  (2022-05-31T03:35:44Z)
• TransEditor: Transformer-Based Dual-Space GAN for Highly Controllable Facial Editing [110.82128064489237]
  We propose TransEditor, a novel Transformer-based framework to enhance interaction in a dual-space GAN for more controllable editing. Experiments demonstrate the superiority of the proposed framework in image quality and editing capability, suggesting the effectiveness of TransEditor for highly controllable facial editing.
  arXiv  Detail & Related papers  (2022-03-31T17:58:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.