Semantic Latent Decomposition with Normalizing Flows for Face Editing

• URL: http://arxiv.org/abs/2309.05314v1
• Date: Mon, 11 Sep 2023 08:59:15 GMT
• Title: Semantic Latent Decomposition with Normalizing Flows for Face Editing
• Abstract summary: Navigating in the latent space of StyleGAN has shown effectiveness for face editing. The resulting methods usually encounter challenges in complicated navigation due to the entanglement among different attributes in the latent space. This paper proposes a novel framework, termed SDFlow, with a semantic decomposition in original latent space.
• Score: 43.41509627246366
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Navigating in the latent space of StyleGAN has shown effectiveness for face editing. However, the resulting methods usually encounter challenges in complicated navigation due to the entanglement among different attributes in the latent space. To address this issue, this paper proposes a novel framework, termed SDFlow, with a semantic decomposition in original latent space using continuous conditional normalizing flows. Specifically, SDFlow decomposes the original latent code into different irrelevant variables by jointly optimizing two components: (i) a semantic encoder to estimate semantic variables from input faces and (ii) a flow-based transformation module to map the latent code into a semantic-irrelevant variable in Gaussian distribution, conditioned on the learned semantic variables. To eliminate the entanglement between variables, we employ a disentangled learning strategy under a mutual information framework, thereby providing precise manipulation controls. Experimental results demonstrate that SDFlow outperforms existing state-of-the-art face editing methods both qualitatively and quantitatively. The source code is made available at https://github.com/phil329/SDFlow.

Related papers

• SeamFlow: Structure-Aware Flow Matching on Edge Probabilities for Artist-Like UV Unwrapping [49.50766584180635]
  We introduce SeamFlow, a novel generative framework for 3D surface cutting.<n>SeamFlow achieves exceptional semantic coherence and remarkably low parameterization distortion.
  arXiv  Detail & Related papers  (2026-09-04T05:31:09Z)
• ChangeFlow -- Latent Rectified Flow for Change Detection in Remote Sensing [0.0]
  ChangeFlow is a generative framework that reformulates change detection as the synthesis of a change mask in latent space via rectified flow.<n>It achieves an average F1 of 80.4%, improving by 1.3 points on average over the previous best method.
  arXiv  Detail & Related papers  (2026-05-14T20:04:16Z)
• SDFlow: Similarity-Driven Flow Matching for Time Series Generation [42.61146250886606]
  We propose SDFlow, a non-autoregressive framework that operates entirely in the frozen VQ latent space.<n>We show that SDFlow achieves state-of-the-art performance, improving Discriminative Score and substantially reducing Context-FID.
  arXiv  Detail & Related papers  (2026-05-07T06:28:18Z)
• ORSIFlow: Saliency-Guided Rectified Flow for Optical Remote Sensing Salient Object Detection [16.437554336020757]
  Optical Remote Sensing Image Salient Object Detection (ORSI-SOD) remains challenging due to complex backgrounds, low contrast, irregular object shapes, and large variations in object scale.<n>We propose ORSIFlow, a saliency-guided rectified flow framework that reformulates ORSI-SOD as a deterministic latent flow generation problem.<n>To enhance saliency awareness, we design a Salient Feature Discriminator for global semantic discrimination and a Salient Feature Calibrator for precise boundary refinement.
  arXiv  Detail & Related papers  (2026-03-30T15:33:26Z)
• Trajectory Stitching for Solving Inverse Problems with Flow-Based Models [68.36374645801901]
  Flow-based generative models have emerged as powerful priors for solving inverse problems.<n>We propose MS-Flow, which represents the trajectory as a sequence of intermediate latent states rather than a single initial code.<n>We demonstrate the effectiveness of MS-Flow over existing methods on image recovery and inverse problems, including inpainting, super-resolution, and computed tomography.
  arXiv  Detail & Related papers  (2026-02-09T11:36:41Z)
• SplitFlow: Flow Decomposition for Inversion-Free Text-to-Image Editing [15.234877788378563]
  Rectified flow models have become a de facto standard in image generation due to their stable sampling trajectories and high-fidelity outputs.<n>Despite their strong generative capabilities, they face critical limitations in image editing tasks.<n>Recent efforts have attempted to directly map source and target distributions via ODE-based approaches without inversion.<n>We propose a flow decomposition-and-aggregation framework built upon an inversion-free formulation to address these limitations.
  arXiv  Detail & Related papers  (2025-10-29T21:12:58Z)
• On the flow matching interpretability [2.816392009888047]
  We propose a framework constraining each flow step to be sampled from a known physical distribution.<n>Flow trajectories are mapped to (and constrained to traverse) the equilibrium states of the simulated physical process.<n>This demonstrates that embedding physical semantics into generative flows transforms neural trajectories into interpretable physical processes.
  arXiv  Detail & Related papers  (2025-10-24T07:26:45Z)
• Style Quantization for Data-Efficient GAN Training [18.40243591024141]
  Under limited data setting, GANs often struggle to navigate and effectively exploit the input latent space. We propose textitSQ-GAN, a novel approach that enhances consistency regularization. Experiments demonstrate significant improvements in both discriminator robustness and generation quality.
  arXiv  Detail & Related papers  (2025-03-31T16:28:44Z)
• Evaluating Semantic Variation in Text-to-Image Synthesis: A Causal Perspective [50.261681681643076]
  We propose a novel metric called SemVarEffect and a benchmark named SemVarBench to evaluate the causality between semantic variations in inputs and outputs in text-to-image synthesis. Our work establishes an effective evaluation framework that advances the T2I synthesis community's exploration of human instruction understanding.
  arXiv  Detail & Related papers  (2024-10-14T08:45:35Z)
• Entropy-Informed Weighting Channel Normalizing Flow [7.751853409569806]
  We propose a regularized and feature-dependent $mathttShuffle$ operation and integrate it into vanilla multi-scale architecture. We observe that such operation guides the variables to evolve in the direction of entropy increase, hence we refer to NFs with the $mathttShuffle$ operation as emphEntropy-Informed Weighting Channel Normalizing Flow (EIW-Flow)
  arXiv  Detail & Related papers  (2024-07-06T04:46:41Z)
• ContextFlow++: Generalist-Specialist Flow-based Generative Models with Mixed-Variable Context Encoding [0.8192907805418583]
  We propose ContextFlow++ to overcome limitations using an additive conditioning with explicit generalist-specialist knowledge decoupling. Our experiments on rotated MNIST-R, corrupted CIFAR-10C, real-world ATM predictive maintenance and SMAP unsupervised anomaly detection benchmarks show that the proposed ContextFlow++ offers faster stable training and achieves higher performance metrics.
  arXiv  Detail & Related papers  (2024-06-02T00:00:00Z)
• PaddingFlow: Improving Normalizing Flows with Padding-Dimensional Noise [4.762593660623934]
  We propose PaddingFlow, a novel dequantization method, which improves normalizing flows with padding-dimensional noise. We validate our method on the main benchmarks of unconditional density estimation. The results show that PaddingFlow can perform better in all experiments in this paper.
  arXiv  Detail & Related papers  (2024-03-13T03:28:39Z)
• Improving the Sample-Complexity of Deep Classification Networks with Invariant Integration [77.99182201815763]
  Leveraging prior knowledge on intraclass variance due to transformations is a powerful method to improve the sample complexity of deep neural networks. We propose a novel monomial selection algorithm based on pruning methods to allow an application to more complex problems. We demonstrate the improved sample complexity on the Rotated-MNIST, SVHN and CIFAR-10 datasets.
  arXiv  Detail & Related papers  (2022-02-08T16:16:11Z)
• Learning Conditional Invariance through Cycle Consistency [60.85059977904014]
  We propose a novel approach to identify meaningful and independent factors of variation in a dataset. Our method involves two separate latent subspaces for the target property and the remaining input information. We demonstrate on synthetic and molecular data that our approach identifies more meaningful factors which lead to sparser and more interpretable models.
  arXiv  Detail & Related papers  (2021-11-25T17:33:12Z)
• Layout-to-Image Translation with Double Pooling Generative Adversarial Networks [76.83075646527521]
  We propose a novel Double Pooing GAN (DPGAN) for generating photo-realistic and semantically-consistent results from the input layout. We also propose a novel Double Pooling Module (DPM), which consists of the Square-shape Pooling Module (SPM) and the Rectangle-shape Pooling Module ( RPM)
  arXiv  Detail & Related papers  (2021-08-29T19:55:14Z)
• Mitigating Generation Shifts for Generalized Zero-Shot Learning [52.98182124310114]
  Generalized Zero-Shot Learning (GZSL) is the task of leveraging semantic information (e.g., attributes) to recognize the seen and unseen samples, where unseen classes are not observable during training. We propose a novel Generation Shifts Mitigating Flow framework for learning unseen data synthesis efficiently and effectively. Experimental results demonstrate that GSMFlow achieves state-of-the-art recognition performance in both conventional and generalized zero-shot settings.
  arXiv  Detail & Related papers  (2021-07-07T11:43:59Z)
• Mask-Guided Discovery of Semantic Manifolds in Generative Models [0.0]
  StyleGAN2 generates images of human faces from random vectors in a lower-dimensional latent space. The model behaves as a black box, providing neither control over its output nor insight into the structures it has learned from the data. We present a method to explore the manifold of changes of spatially localized regions of the face.
  arXiv  Detail & Related papers  (2021-05-15T18:06:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.