Functional Diffusion

• URL: http://arxiv.org/abs/2311.15435v1
• Date: Sun, 26 Nov 2023 21:35:34 GMT
• Title: Functional Diffusion
• Authors: Biao Zhang and Peter Wonka
• Abstract summary: We propose a new class of generative diffusion models, called functional diffusion. functional diffusion can be seen as an extension of classical diffusion models to an infinite-dimensional domain. We show generative results on complicated signed distance functions and deformation functions defined on 3D surfaces.
• Score: 55.251174506648454
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a new class of generative diffusion models, called functional diffusion. In contrast to previous work, functional diffusion works on samples that are represented by functions with a continuous domain. Functional diffusion can be seen as an extension of classical diffusion models to an infinite-dimensional domain. Functional diffusion is very versatile as images, videos, audio, 3D shapes, deformations, \etc, can be handled by the same framework with minimal changes. In addition, functional diffusion is especially suited for irregular data or data defined in non-standard domains. In our work, we derive the necessary foundations for functional diffusion and propose a first implementation based on the transformer architecture. We show generative results on complicated signed distance functions and deformation functions defined on 3D surfaces.

Related papers

• Suppress Content Shift: Better Diffusion Features via Off-the-Shelf Generation Techniques [119.02857688205295]
  We discover that diffusion feature has been hindered by a hidden yet universal phenomenon that we call content shift. We locate the cause of content shift as one inherent characteristic of diffusion models, which suggests the broad existence of this phenomenon in diffusion feature. We propose a practical guideline named GATE to efficiently evaluate the potential benefit of a technique and provide an implementation of our methodology.
  arXiv  Detail & Related papers  (2024-10-09T09:43:36Z)
• G2D2: Gradient-guided Discrete Diffusion for image inverse problem solving [55.185588994883226]
  This paper presents a novel method for addressing linear inverse problems by leveraging image-generation models based on discrete diffusion as priors. To the best of our knowledge, this is the first approach to use discrete diffusion model-based priors for solving image inverse problems.
  arXiv  Detail & Related papers  (2024-10-09T06:18:25Z)
• Text-to-Image Rectified Flow as Plug-and-Play Priors [52.586838532560755]
  Rectified flow is a novel class of generative models that enforces a linear progression from the source to the target distribution. We show that rectified flow approaches surpass in terms of generation quality and efficiency, requiring fewer inference steps. Our method also displays competitive performance in image inversion and editing.
  arXiv  Detail & Related papers  (2024-06-05T14:02:31Z)
• Stochastic Optimal Control for Diffusion Bridges in Function Spaces [13.544676987441441]
  We present a theory of optimal control tailored to infinite-dimensional spaces. We show how Doob's $h$-transform can be derived from the SOC perspective and expanded to infinite dimensions. We propose two applications: learning bridges between two infinite-dimensional distributions and generative models for sampling from an infinite-dimensional distribution.
  arXiv  Detail & Related papers  (2024-05-31T05:42:47Z)
• Guided Diffusion from Self-Supervised Diffusion Features [49.78673164423208]
  Guidance serves as a key concept in diffusion models, yet its effectiveness is often limited by the need for extra data annotation or pretraining. We propose a framework to extract guidance from, and specifically for, diffusion models.
  arXiv  Detail & Related papers  (2023-12-14T11:19:11Z)
• Swallowing the Bitter Pill: Simplified Scalable Conformer Generation [12.341835649897886]
  We present a novel way to predict molecular conformers through a simple formulation that sidesteps many of the equis of prior works and achieves state of the art results by using the advantages of scale. We are able to radically simplify structure learning, and make it trivial to scale up the model sizes. This model, called Molecular Conformer Fields (MCF), works by parameterizing conformer structures as functions that map elements from a molecular graph directly to their 3D location in space.
  arXiv  Detail & Related papers  (2023-11-27T22:53:41Z)
• Manifold Diffusion Fields [11.4726574705951]
  We present an approach that unlocks learning of diffusion models of data in non-Euclidean geometries. We define an intrinsic coordinate system on the manifold via the eigen-functions of the Laplace-Beltrami Operator. We show that MDF can capture distributions of such functions with better diversity and fidelity than previous approaches.
  arXiv  Detail & Related papers  (2023-05-24T21:42:45Z)
• Diffusion Probabilistic Fields [42.428882785136295]
  We introduce Diffusion Probabilistic Fields (DPF), a diffusion model that can learn distributions over continuous functions defined over metric spaces. We empirically show that DPF effectively deals with different modalities like 2D images and 3D geometry, in addition to modeling distributions over fields defined on non-Euclidean metric spaces.
  arXiv  Detail & Related papers  (2023-03-01T01:37:24Z)
• Diffusion Generative Models in Infinite Dimensions [10.15736468214228]
  We generalize diffusion generative models to operate directly in function space. A significant benefit of our function space point of view is that it allows us to explicitly specify the space of functions we are working in. Our approach allows us to perform both unconditional and conditional generation of function-valued data.
  arXiv  Detail & Related papers  (2022-12-01T21:54:19Z)
• Unifying Diffusion Models' Latent Space, with Applications to CycleDiffusion and Guidance [95.12230117950232]
  We show that a common latent space emerges from two diffusion models trained independently on related domains. Applying CycleDiffusion to text-to-image diffusion models, we show that large-scale text-to-image diffusion models can be used as zero-shot image-to-image editors.
  arXiv  Detail & Related papers  (2022-10-11T15:53:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.