CaricHarmony: Contrastive Diffusion Paths for Identity-Preserving Caricature Synthesis

• URL: http://arxiv.org/abs/2606.13964v1
• Date: Thu, 11 Jun 2026 22:57:59 GMT
• Title: CaricHarmony: Contrastive Diffusion Paths for Identity-Preserving Caricature Synthesis
• Abstract summary: Sketch-based caricature synthesis suffers from a fundamental failure mode.<n>Identity and shape conditions are combined in diffusion models, causing collapse toward bland portraits or unrecognizable distortions.<n>We present CaricHarmony, the first training-free method that explicitly resolves this contamination through parallel uncontaminated diffusion paths.
• Score: 49.596677723190886
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Sketch-based caricature synthesis suffers from a fundamental failure mode: when identity and shape conditions are combined in diffusion models, they create destructive interference that causes inevitable collapse toward either bland portraits or unrecognizable distortions. We identify the root cause as \emph{condition signal contamination} -- competing probability distributions in the denoising trajectory that make balanced generation impossible. We present CaricHarmony, the first training-free method that explicitly resolves this contamination through parallel uncontaminated diffusion paths. During inference, we maintain three paths: $\mathcal{P}^{\mathrm{i}}$ (pure identity), $\mathcal{P}^{\mathrm{s}}$ (pure shape), and $\mathcal{P}^{\mathrm{i+s}}$ (harmonized output). Novel energy functions operating on cross-attention features provide gradient guidance that steers $\mathcal{P}^{\mathrm{i+s}}$ toward optimal balance: $\mathcal{E}_{\mathrm{shape}}$ ensures sketch fidelity through layout and semantic alignment, while $\mathcal{E}_{\mathrm{id}}$ employs token-level correspondence matching robust to extreme distortions. Unlike DemoCaricature requiring 70 seconds per-identity fine-tuning or CaricatureBooth constrained to Bezier curves, CaricHarmony accepts any sketch format and generates in under 16 seconds. Experiments demonstrate state-of-the-art performance: 0.8615 shape CLIP score (vs. 0.8450) under comparable identity consistency score, with 7.81 overall user preference score (vs. 6.06). Our method fundamentally reconceptualizes the ID-shape conflict as conditioning signal contamination for diffusion models, enabling unprecedented creative control while preserving recognition.

Related papers

• Harmfulness Propagation Dynamics: Layer-wise Trajectories of Adversarial Intent in Large Language Models [0.0]
  textbfherald (textbfEncoding textbfRecognition via textbfActivation textbfLayer textbfDynamics)<n>textbfherald (textbfHarmful textbfEncoding textbfRecognition via textbfActivation textbfLayer textbfDynamics)<n>
  arXiv  Detail & Related papers  (2026-09-11T21:04:55Z)
• Drift Variation Autoencoder: Unifying Generation and Representation Learning through Conditional Posterior Flow Matching [0.0]
  This work takes the corresponding posterior $P(Xmid C)$ as the common statistical object for conditional generation.<n> Drift Variation autoencoder trains a masked encoder $Z=E(C)$ and a conditional flow decoder with one clean-prediction Flow Matching loss.
  arXiv  Detail & Related papers  (2026-08-25T20:38:50Z)
• Prism: Structural Symmetry Scanning via Duality-Constrained Laplacian Projection [0.0]
  We introduce textbfPrism, a framework for structural symmetry diagnosis in complex networks.<n>Prism computes the emphduality defect $(L,P) = |LP - PL|_F / |L|_F$ -- a scalar measuring how far the network deviates from structural self-consistency.
  arXiv  Detail & Related papers  (2026-05-18T04:20:05Z)
• The Geometry of Forgetting: Temporal Knowledge Drift as an Independent Axis in LLM Representations [50.43168858368539]
  Large language models confidently produce outdated answers, and no existing method can detect them.<n>We show this is not an engineering failure but a structural one: temporal drift is encoded as a direction in the residual stream geometrically to both correctness and uncertainty.
  arXiv  Detail & Related papers  (2026-05-09T22:27:31Z)
• Why Training-Free Token Reduction Collapses: The Inherent Instability of Pairwise Scoring Signals [0.0]
  Training-free token reduction methods for Vision Transformers employ different scoring mechanisms, yet they share a closely matched cliff-like collapse at high compression.<n>We develop a diagnostic framework with two tools, ranking consistency $_s$ and off-diagonal correlation $_textoff$, that decomposes the collapse into (1)a signal-agnostic error amplifier inherent to layer-wise reduction.<n>We construct CATIS as a constructive validation: unary signals raise the trigger threshold, triage suppresses the gain.
  arXiv  Detail & Related papers  (2026-04-17T23:26:27Z)
• INC: An Indirect Neural Corrector for Auto-Regressive Hybrid PDE Solvers [61.84396402100827]
  We propose the Indirect Neural Corrector ($mathrmINC$), which integrates learned corrections into the governing equations.<n>$mathrmINC$ reduces the error amplification on the order of $t-1 + L$, where $t$ is the timestep and $L$ the Lipschitz constant.<n>We test $mathrmINC$ in extensive benchmarks, covering numerous differentiable solvers, neural backbones, and test cases ranging from a 1D chaotic system to 3D turbulence.
  arXiv  Detail & Related papers  (2025-11-16T20:14:28Z)
• Coupled Data and Measurement Space Dynamics for Enhanced Diffusion Posterior Sampling [27.146380722473932]
  Inverse problems, where the goal is to recover an unknown signal from noisy or incomplete measurements, are central to medical imaging, remote sensing, and computational biology.<n>We propose a novel framework called emphcoupled data and space diffusion posterior sampling (C-DPS), which eliminates the need for constraint tuning or likelihood measurement.<n>C-DPS consistently outperforms existing baselines, both qualitatively and quantitatively, across multiple inverse problem benchmarks.
  arXiv  Detail & Related papers  (2025-10-08T18:59:16Z)
• Robust Graph-Based Semi-Supervised Learning via $p$-Conductances [49.0776396776252]
  We study the problem of semi-supervised learning on graphs in the regime where data labels are scarce or possibly corrupted.<n>We propose an approach called $p$-conductance learning that generalizes the $p$-Laplace and Poisson learning methods.<n> Empirical results on computer vision and citation datasets demonstrate that our approach achieves state-of-the-art accuracy in low label-rate, corrupted-label, and partial-label regimes.
  arXiv  Detail & Related papers  (2025-02-13T01:11:25Z)
• Self-Ensembling Gaussian Splatting for Few-Shot Novel View Synthesis [55.561961365113554]
  3D Gaussian Splatting (3DGS) has demonstrated remarkable effectiveness in novel view synthesis (NVS)<n>In this paper, we introduce Self-Ensembling Gaussian Splatting (SE-GS)<n>We achieve self-ensembling by incorporating an uncertainty-aware perturbation strategy during training.<n> Experimental results on the LLFF, Mip-NeRF360, DTU, and MVImgNet datasets demonstrate that our approach enhances NVS quality under few-shot training conditions.
  arXiv  Detail & Related papers  (2024-10-31T18:43:48Z)
• Convergence Analysis of Probability Flow ODE for Score-based Generative Models [5.939858158928473]
  We study the convergence properties of deterministic samplers based on probability flow ODEs from both theoretical and numerical perspectives.<n>We prove the total variation between the target and the generated data distributions can be bounded above by $mathcalO(d3/4delta1/2)$ in the continuous time level.
  arXiv  Detail & Related papers  (2024-04-15T12:29:28Z)
• Releasing Inequality Phenomenon in $\ell_{\infty}$-norm Adversarial Training via Input Gradient Distillation [66.5912840038179]
  A recent study revealed that (ell_infty)-norm adversarial training ((ell_infty)-AT) will induce unevenly distributed input gradients.<n>This phenomenon makes the (ell_infty)-norm adversarially trained model more vulnerable than the standard-trained model.<n>We propose a simple yet effective method called Input Gradient Distillation (IGD) to release the inequality phenomenon in $ell_infty$-AT.
  arXiv  Detail & Related papers  (2023-05-16T09:23:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.