FFaceNeRF: Few-shot Face Editing in Neural Radiance Fields

• URL: http://arxiv.org/abs/2503.17095v1
• Date: Fri, 21 Mar 2025 12:24:58 GMT
• Title: FFaceNeRF: Few-shot Face Editing in Neural Radiance Fields
• Authors: Kwan Yun, Chaelin Kim, Hangyeul Shin, Junyong Noh,
• Abstract summary: Recent 3D face editing methods using masks have produced high-quality edited images by leveraging Neural Radiance Fields (NeRF)<n>We present FFaceNeRF, a NeRF-based face editing technique that can overcome the challenge of limited user control due to the use of fixed mask layouts.
• Score: 4.599829659954009
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Recent 3D face editing methods using masks have produced high-quality edited images by leveraging Neural Radiance Fields (NeRF). Despite their impressive performance, existing methods often provide limited user control due to the use of pre-trained segmentation masks. To utilize masks with a desired layout, an extensive training dataset is required, which is challenging to gather. We present FFaceNeRF, a NeRF-based face editing technique that can overcome the challenge of limited user control due to the use of fixed mask layouts. Our method employs a geometry adapter with feature injection, allowing for effective manipulation of geometry attributes. Additionally, we adopt latent mixing for tri-plane augmentation, which enables training with a few samples. This facilitates rapid model adaptation to desired mask layouts, crucial for applications in fields like personalized medical imaging or creative face editing. Our comparative evaluations demonstrate that FFaceNeRF surpasses existing mask based face editing methods in terms of flexibility, control, and generated image quality, paving the way for future advancements in customized and high-fidelity 3D face editing. The code is available on the {\href{https://kwanyun.github.io/FFaceNeRF_page/}{project-page}}.

Related papers

• Revealing Directions for Text-guided 3D Face Editing [52.85632020601518]
  3D face editing is a significant task in multimedia, aimed at the manipulation of 3D face models across various control signals. We present Face Clan, a text-general approach for generating and manipulating 3D faces based on arbitrary attribute descriptions. Our method offers a precisely controllable manipulation method, allowing users to intuitively customize regions of interest with the text description.
  arXiv  Detail & Related papers  (2024-10-07T12:04:39Z)
• Face Mask Removal with Region-attentive Face Inpainting [0.7433327915285965]
  We propose a generative face inpainting method to recover/reconstruct the masked part of a face. Our proposed method includes a Multi-scale Channel-Spatial Attention Module (M-CSAM) to mitigate the spatial information loss. We synthesize our own Masked-Faces dataset from the CelebA dataset by incorporating five different types of face masks.
  arXiv  Detail & Related papers  (2024-09-10T20:10:11Z)
• NeRF-Insert: 3D Local Editing with Multimodal Control Signals [97.91172669905578]
  NeRF-Insert is a NeRF editing framework that allows users to make high-quality local edits with a flexible level of control. We cast scene editing as an in-painting problem, which encourages the global structure of the scene to be preserved. Our results show better visual quality and also maintain stronger consistency with the original NeRF.
  arXiv  Detail & Related papers  (2024-04-30T02:04:49Z)
• MaTe3D: Mask-guided Text-based 3D-aware Portrait Editing [61.014328598895524]
  We propose textbfMaTe3D: mask-guided text-based 3D-aware portrait editing. New SDF-based 3D generator learns local and global representations with proposed SDF and density consistency losses. Conditional Distillation on Geometry and Texture (CDGT) mitigates visual ambiguity and avoids mismatch between texture and geometry.
  arXiv  Detail & Related papers  (2023-12-12T03:04:08Z)
• FaceDNeRF: Semantics-Driven Face Reconstruction, Prompt Editing and Relighting with Diffusion Models [67.17713009917095]
  We propose Face Diffusion NeRF (FaceDNeRF), a new generative method to reconstruct high-quality Face NeRFs from single images. With carefully designed illumination and identity preserving loss, FaceDNeRF offers users unparalleled control over the editing process.
  arXiv  Detail & Related papers  (2023-06-01T15:14:39Z)
• NeRFFaceEditing: Disentangled Face Editing in Neural Radiance Fields [40.543998582101146]
  We introduce NeRFFaceEditing, which enables editing and decoupling geometry and appearance in neural radiance fields. Our method allows users to edit via semantic masks with decoupled control of geometry and appearance. Both qualitative and quantitative evaluations show the superior geometry and appearance control abilities of our method compared to existing and alternative solutions.
  arXiv  Detail & Related papers  (2022-11-15T08:11:39Z)
• HiMFR: A Hybrid Masked Face Recognition Through Face Inpainting [0.7868449549351486]
  We propose an end-to-end hybrid masked face recognition system, namely HiMFR. Masked face detector module applies a pretrained Vision Transformer to detect whether faces are covered with masked or not. Inpainting module uses a fine-tune image inpainting model based on a Generative Adversarial Network (GAN) to restore faces. Finally, the hybrid face recognition module based on ViT with an EfficientNetB3 backbone recognizes the faces.
  arXiv  Detail & Related papers  (2022-09-19T11:26:49Z)
• Layered Depth Refinement with Mask Guidance [61.10654666344419]
  We formulate a novel problem of mask-guided depth refinement that utilizes a generic mask to refine the depth prediction of SIDE models. Our framework performs layered refinement and inpainting/outpainting, decomposing the depth map into two separate layers signified by the mask and the inverse mask. We empirically show that our method is robust to different types of masks and initial depth predictions, accurately refining depth values in inner and outer mask boundary regions.
  arXiv  Detail & Related papers  (2022-06-07T06:42:44Z)
• Contrastive Context-Aware Learning for 3D High-Fidelity Mask Face Presentation Attack Detection [103.7264459186552]
  Face presentation attack detection (PAD) is essential to secure face recognition systems. Most existing 3D mask PAD benchmarks suffer from several drawbacks. We introduce a largescale High-Fidelity Mask dataset to bridge the gap to real-world applications.
  arXiv  Detail & Related papers  (2021-04-13T12:48:38Z)
• A 3D model-based approach for fitting masks to faces in the wild [9.958467179573235]
  We present a 3D model-based approach called WearMask3D for augmenting face images of various poses to the masked face counterparts. Our method proceeds by first fitting a 3D morphable model on the input image, second overlaying the mask surface onto the face model and warping the respective mask texture, and last projecting the 3D mask back to 2D. Experimental results demonstrate WearMask3D produces more realistic masked images, and utilizing these images for training leads to improved recognition accuracy of masked faces.
  arXiv  Detail & Related papers  (2021-03-01T06:50:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.