Training-Free Constrained Generation With Stable Diffusion Models

• URL: http://arxiv.org/abs/2502.05625v3
• Date: Fri, 06 Jun 2025 20:42:50 GMT
• Title: Training-Free Constrained Generation With Stable Diffusion Models
• Authors: Stefano Zampini, Jacob K. Christopher, Luca Oneto, Davide Anguita, Ferdinando Fioretto,
• Abstract summary: Stable diffusion models represent the state-of-the-art in data synthesis across diverse domains.<n>This paper proposes a novel integration of stable diffusion models with constrained optimization frameworks.<n>The effectiveness of this approach is demonstrated through material design experiments requiring adherence to precise morphometric properties.
• Score: 45.138721047543214
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Stable diffusion models represent the state-of-the-art in data synthesis across diverse domains and hold transformative potential for applications in science and engineering, e.g., by facilitating the discovery of novel solutions and simulating systems that are computationally intractable to model explicitly. While there is increasing effort to incorporate physics-based constraints into generative models, existing techniques are either limited in their applicability to latent diffusion frameworks or lack the capability to strictly enforce domain-specific constraints. To address this limitation this paper proposes a novel integration of stable diffusion models with constrained optimization frameworks, enabling the generation of outputs satisfying stringent physical and functional requirements. The effectiveness of this approach is demonstrated through material design experiments requiring adherence to precise morphometric properties, challenging inverse design tasks involving the generation of materials inducing specific stress-strain responses, and copyright-constrained content generation tasks.

Related papers

• Strictly Constrained Generative Modeling via Split Augmented Langevin Sampling [0.1052166918701117]
  We develop a principled framework for sampling from a target distribution while rigorously satisfying physical constraints.<n>We propose Split Augmented Langevin (SAL), a novel primal-dual sampling algorithm that enforces constraints progressively through variable splitting, with convergence guarantees.<n>We apply our method to diffusion-based data assimilation on a complex physical system, where enforcing physical constraints substantially improves both forecast accuracy and the preservation of critical conserved quantities.
  arXiv  Detail & Related papers  (2025-05-23T15:21:10Z)
• Constrained Language Generation with Discrete Diffusion Models [61.81569616239755]
  We present Constrained Discrete Diffusion (CDD), a novel method for enforcing constraints on natural language by integrating discrete diffusion models with differentiable optimization. We show how this technique can be applied to satisfy a variety of natural language constraints, including (i) toxicity mitigation by preventing harmful content from emerging, (ii) character and sequence level lexical constraints, and (iii) novel molecule sequence generation with specific property adherence.
  arXiv  Detail & Related papers  (2025-03-12T19:48:12Z)
• Efficient Fine-Tuning and Concept Suppression for Pruned Diffusion Models [93.76814568163353]
  We propose a novel bilevel optimization framework for pruned diffusion models. This framework consolidates the fine-tuning and unlearning processes into a unified phase. It is compatible with various pruning and concept unlearning methods.
  arXiv  Detail & Related papers  (2024-12-19T19:13:18Z)
• Hard Constraint Guided Flow Matching for Gradient-Free Generation of PDE Solutions [41.558608119074755]
  We introduce a novel framework for adapting pre-trained, unconstrained flow-matching models to satisfy constraints exactly in a zero-shot manner without requiring expensive computations or fine-tuning.<n>Our framework, ECI sampling, alternates between extrapolation (E), correction (C) and Generative (I) stages to ensure accurate integration of constraint information while preserving the validity of the generation.<n>We demonstrate the effectiveness of our approach across various PDE systems, showing that ECI-guided generation strictly adheres to physical constraints and accurately captures complex distribution shifts induced by these constraints.
  arXiv  Detail & Related papers  (2024-12-02T18:36:26Z)
• Accounting for plasticity: An extension of inelastic Constitutive Artificial Neural Networks [0.0]
  We extend the existing framework of inelastic artificial neural networks (iCANNs) by incorporating plasticity to increase their applicability to model more complex material behavior.<n>Our framework captures both linear and nonlinear kinematic hardening behavior.
  arXiv  Detail & Related papers  (2024-07-27T19:19:42Z)
• Constrained Synthesis with Projected Diffusion Models [47.56192362295252]
  This paper introduces an approach to generative diffusion processes the ability to satisfy and certify compliance with constraints and physical principles. The proposed method recast the traditional process of generative diffusion as a constrained distribution problem to ensure adherence to constraints.
  arXiv  Detail & Related papers  (2024-02-05T22:18:16Z)
• CoCoGen: Physically-Consistent and Conditioned Score-based Generative Models for Forward and Inverse Problems [1.0923877073891446]
  This work extends the reach of generative models into physical problem domains. We present an efficient approach to promote consistency with the underlying PDE. We showcase the potential and versatility of score-based generative models in various physics tasks.
  arXiv  Detail & Related papers  (2023-12-16T19:56:10Z)
• Discovering Interpretable Physical Models using Symbolic Regression and Discrete Exterior Calculus [55.2480439325792]
  We propose a framework that combines Symbolic Regression (SR) and Discrete Exterior Calculus (DEC) for the automated discovery of physical models. DEC provides building blocks for the discrete analogue of field theories, which are beyond the state-of-the-art applications of SR to physical problems. We prove the effectiveness of our methodology by re-discovering three models of Continuum Physics from synthetic experimental data.
  arXiv  Detail & Related papers  (2023-10-10T13:23:05Z)
• Extreme sparsification of physics-augmented neural networks for interpretable model discovery in mechanics [0.0]
  We propose to train regularized physics-augmented neural network-based models utilizing a smoothed version of $L0$-regularization. We show that the method can reliably obtain interpretable and trustworthy models for compressible and incompressible thermodynamicity, yield functions, and hardening models for elastoplasticity.
  arXiv  Detail & Related papers  (2023-10-05T16:28:58Z)
• Diffusion Models for Constrained Domains [11.488860260925504]
  We present two distinct noising processes based on (i) the logarithmic barrier metric and (ii) the reflected Brownian motion induced by the constraints. We then demonstrate the practical utility of our methods on a number of synthetic and real-world tasks, including applications from robotics and protein design.
  arXiv  Detail & Related papers  (2023-04-11T17:19:45Z)
• Neural Abstractions [72.42530499990028]
  We present a novel method for the safety verification of nonlinear dynamical models that uses neural networks to represent abstractions of their dynamics. We demonstrate that our approach performs comparably to the mature tool Flow* on existing benchmark nonlinear models.
  arXiv  Detail & Related papers  (2023-01-27T12:38:09Z)
• Latent Diffusion Energy-Based Model for Interpretable Text Modeling [104.85356157724372]
  We introduce a novel symbiosis between the diffusion models and latent space EBMs in a variational learning framework. We develop a geometric clustering-based regularization jointly with the information bottleneck to further improve the quality of the learned latent space.
  arXiv  Detail & Related papers  (2022-06-13T03:41:31Z)
• A data-driven peridynamic continuum model for upscaling molecular dynamics [3.1196544696082613]
  We propose a learning framework to extract, from molecular dynamics data, an optimal Linear Peridynamic Solid model. We provide sufficient well-posedness conditions for discretized LPS models with sign-changing influence functions. This framework guarantees that the resulting model is mathematically well-posed, physically consistent, and that it generalizes well to settings that are different from the ones used during training.
  arXiv  Detail & Related papers  (2021-08-04T07:07:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.