Advanced Image Generation: Negative Prompt Optimization and Latent Classifier Guidance

• URL: http://arxiv.org/abs/2607.14580v1
• Date: Thu, 16 Jul 2026 05:21:17 GMT
• Title: Advanced Image Generation: Negative Prompt Optimization and Latent Classifier Guidance
• Abstract summary: We present a novel system that integrates negative prompt optimization via a fine-tuned sequence-to-sequence LLM and latent-space classifier guidance to improve the quality of images generated by Stable Diffusion.<n>Our approach automatically generates optimized negative prompts, and employs a CNN-RNN hybrid classifier to evaluate and guide diffusion steps, rolling back low-quality latent updates.<n> Experimental results demonstrate that our dual-guidance framework reduces artifacts and improves semantic fidelity compared to baseline diffusion.
• Score: 0.0815557531820863
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present a novel system that integrates negative prompt optimization via a fine-tuned sequence-to-sequence LLM and latent-space classifier guidance to improve the quality of images generated by Stable Diffusion. Our approach automatically generates optimized negative prompts, and employs a CNN-RNN hybrid classifier to evaluate and guide diffusion steps, rolling back low-quality latent updates. Experimental results demonstrate that our dual-guidance framework reduces artifacts and improves semantic fidelity compared to baseline diffusion.

Related papers

• Prompt Optimization Via Diffusion Language Models [73.9599434962714]
  We propose a diffusion-based framework for prompt optimization.<n>Our method enables flexible, span-level prompt updates without requiring access or modifying the downstream language model.<n>We show that moderate diffusion step counts provide the best balance between refinement quality and stability.
  arXiv  Detail & Related papers  (2026-01-30T00:00:54Z)
• Enhancing Diffusion Model Guidance through Calibration and Regularization [9.22066257345387]
  This paper introduces two complementary contributions to address this issue.<n>First, we propose a differentiable calibration objective based on the Smooth Expected Error (Smooth ECE)<n>Second, we develop enhanced sampling guidance methods that operate on off-the-shelf classifiers without requiring retraining.
  arXiv  Detail & Related papers  (2025-11-08T04:23:42Z)
• Knowledge Regularized Negative Feature Tuning of Vision-Language Models for Out-of-Distribution Detection [54.433899174017185]
  Out-of-distribution (OOD) detection is crucial for building reliable machine learning models.<n>We propose a novel method called Knowledge Regularized Negative Feature Tuning (KR-NFT)<n>NFT applies distribution-aware transformations to pre-trained text features, effectively separating positive and negative features into distinct spaces.<n>When trained with few-shot samples from ImageNet dataset, KR-NFT not only improves ID classification accuracy and OOD detection but also significantly reduces the FPR95 by 5.44%.
  arXiv  Detail & Related papers  (2025-07-26T07:44:04Z)
• Negative-Guided Subject Fidelity Optimization for Zero-Shot Subject-Driven Generation [52.18071720309418]
  We present Subject Fidelity Optimization (SFO), a novel comparative learning framework for zero-shot subject-driven generation.<n>SFO guides the model to favor positives over negatives through pairwise comparison.<n>For negative targets, we propose Condition-Degradation Negative Sampling (CDNS), which automatically produces synthetic negatives tailored for subject-driven generation.
  arXiv  Detail & Related papers  (2025-06-04T06:59:25Z)
• HiFi Tuner: High-Fidelity Subject-Driven Fine-Tuning for Diffusion Models [56.112302700630806]
  We introduce an innovative algorithm named HiFi Tuner to enhance the appearance preservation of objects during personalized image generation. Key enhancements include the utilization of mask guidance, a novel parameter regularization technique, and the incorporation of step-wise subject representations. We extend our method to a novel image editing task: substituting the subject in an image through textual manipulations.
  arXiv  Detail & Related papers  (2023-11-30T02:33:29Z)
• Contrast-augmented Diffusion Model with Fine-grained Sequence Alignment for Markup-to-Image Generation [15.411325887412413]
  This paper proposes a novel model named "Contrast-augmented Diffusion Model with Fine-grained Sequence Alignment" (FSA-CDM) FSA-CDM introduces contrastive positive/negative samples into the diffusion model to boost performance for markup-to-image generation. Experiments are conducted on four benchmark datasets from different domains.
  arXiv  Detail & Related papers  (2023-08-02T13:43:03Z)
• Conditional Denoising Diffusion for Sequential Recommendation [62.127862728308045]
  Two prominent generative models, Generative Adversarial Networks (GANs) and Variational AutoEncoders (VAEs) GANs suffer from unstable optimization, while VAEs are prone to posterior collapse and over-smoothed generations. We present a conditional denoising diffusion model, which includes a sequence encoder, a cross-attentive denoising decoder, and a step-wise diffuser.
  arXiv  Detail & Related papers  (2023-04-22T15:32:59Z)
• End-to-End Diffusion Latent Optimization Improves Classifier Guidance [81.27364542975235]
  Direct Optimization of Diffusion Latents (DOODL) is a novel guidance method. It enables plug-and-play guidance by optimizing diffusion latents. It outperforms one-step classifier guidance on computational and human evaluation metrics.
  arXiv  Detail & Related papers  (2023-03-23T22:43:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.