Diffusion priors for Bayesian 3D reconstruction from incomplete measurements

• URL: http://arxiv.org/abs/2412.14897v1
• Date: Thu, 19 Dec 2024 14:28:00 GMT
• Title: Diffusion priors for Bayesian 3D reconstruction from incomplete measurements
• Authors: Julian L. Möbius, Michael Habeck,
• Abstract summary: We explore the use of diffusion models as priors combined with experimental data within a Bayesian framework. We train diffusion models that generate coarse-grained 3D structures at a medium resolution and integrate these with incomplete and noisy experimental data. We find that posterior sampling with diffusion model priors allows for 3D reconstruction from very sparse, low-resolution and partial observations.
• Score: 0.0
• License:
• Abstract: Many inverse problems are ill-posed and need to be complemented by prior information that restricts the class of admissible models. Bayesian approaches encode this information as prior distributions that impose generic properties on the model such as sparsity, non-negativity or smoothness. However, in case of complex structured models such as images, graphs or three-dimensional (3D) objects,generic prior distributions tend to favor models that differ largely from those observed in the real world. Here we explore the use of diffusion models as priors that are combined with experimental data within a Bayesian framework. We use 3D point clouds to represent 3D objects such as household items or biomolecular complexes formed from proteins and nucleic acids. We train diffusion models that generate coarse-grained 3D structures at a medium resolution and integrate these with incomplete and noisy experimental data. To demonstrate the power of our approach, we focus on the reconstruction of biomolecular assemblies from cryo-electron microscopy (cryo-EM) images, which is an important inverse problem in structural biology. We find that posterior sampling with diffusion model priors allows for 3D reconstruction from very sparse, low-resolution and partial observations.

Related papers

• DSplats: 3D Generation by Denoising Splats-Based Multiview Diffusion Models [67.50989119438508]
  We introduce DSplats, a novel method that directly denoises multiview images using Gaussian-based Reconstructors to produce realistic 3D assets. Our experiments demonstrate that DSplats not only produces high-quality, spatially consistent outputs, but also sets a new standard in single-image to 3D reconstruction.
  arXiv  Detail & Related papers  (2024-12-11T07:32:17Z)
• Equivariant Blurring Diffusion for Hierarchical Molecular Conformer Generation [18.394348744611662]
  We introduce a novel generative model termed Equivariant Blurring Diffusion (EBD) EBD defines a forward process that moves towards the fragment-level coarse-grained structure by blurring the fine atomic details of conformers. We demonstrate the effectiveness of EBD by geometric and chemical comparison to state-of-the-art denoising diffusion models on a benchmark of drug-like molecules.
  arXiv  Detail & Related papers  (2024-10-26T19:17:31Z)
• Unsupervised 3D Pose Estimation with Non-Rigid Structure-from-Motion Modeling [83.76377808476039]
  We propose a new modeling method for human pose deformations and design an accompanying diffusion-based motion prior. Inspired by the field of non-rigid structure-from-motion, we divide the task of reconstructing 3D human skeletons in motion into the estimation of a 3D reference skeleton. A mixed spatial-temporal NRSfMformer is used to simultaneously estimate the 3D reference skeleton and the skeleton deformation of each frame from 2D observations sequence.
  arXiv  Detail & Related papers  (2023-08-18T16:41:57Z)
• Reference-Free Isotropic 3D EM Reconstruction using Diffusion Models [8.590026259176806]
  We propose a diffusion-model-based framework that overcomes the limitations of requiring reference data or prior knowledge about the degradation process. Our approach utilizes 2D diffusion models to consistently reconstruct 3D volumes and is well-suited for highly downsampled data.
  arXiv  Detail & Related papers  (2023-08-03T07:57:02Z)
• Geometric Latent Diffusion Models for 3D Molecule Generation [172.15028281732737]
  Generative models, especially diffusion models (DMs), have achieved promising results for generating feature-rich geometries. We propose a novel and principled method for 3D molecule generation named Geometric Latent Diffusion Models (GeoLDM)
  arXiv  Detail & Related papers  (2023-05-02T01:07:22Z)
• 3D Equivariant Diffusion for Target-Aware Molecule Generation and Affinity Prediction [9.67574543046801]
  The inclusion of 3D structures during targeted drug design shows superior performance to other target-free models. We develop a 3D equivariant diffusion model to solve the above challenges. Our model could generate molecules with more realistic 3D structures and better affinities towards the protein targets, and improve binding affinity ranking and prediction without retraining.
  arXiv  Detail & Related papers  (2023-03-06T23:01:43Z)
• Latent Space Diffusion Models of Cryo-EM Structures [6.968705314671148]
  We train a diffusion model as an expressive, learnable prior in the cryoDRGN framework. By learning an accurate model of the data distribution, our method unlocks tools in generative modeling, sampling, and distribution analysis.
  arXiv  Detail & Related papers  (2022-11-25T15:17:10Z)
• A Diffusion Model Predicts 3D Shapes from 2D Microscopy Images [36.17295590429516]
  We introduce DISPR, a diffusion-based model for solving the inverse problem of 3D cell shape prediction from 2D microscopy images. We demonstrate that diffusion models can be applied to inverse problems in 3D, and that they learn to reconstruct 3D shapes with realistic morphological features from 2D microscopy images.
  arXiv  Detail & Related papers  (2022-08-30T10:21:40Z)
• Equivariant Diffusion for Molecule Generation in 3D [74.289191525633]
  This work introduces a diffusion model for molecule computation generation in 3D that is equivariant to Euclidean transformations. Experimentally, the proposed method significantly outperforms previous 3D molecular generative methods regarding the quality of generated samples and efficiency at training time.
  arXiv  Detail & Related papers  (2022-03-31T12:52:25Z)
• 3D Reconstruction of Curvilinear Structures with Stereo Matching DeepConvolutional Neural Networks [52.710012864395246]
  We propose a fully automated pipeline for both detection and matching of curvilinear structures in stereo pairs. We mainly focus on 3D reconstruction of dislocations from stereo pairs of TEM images.
  arXiv  Detail & Related papers  (2021-10-14T23:05:47Z)
• Data-Driven Shadowgraph Simulation of a 3D Object [50.591267188664666]
  We are replacing the numerical code by a computationally cheaper projection based surrogate model. The model is able to approximate the electric fields at a given time without computing all preceding electric fields as required by numerical methods. This model has shown a good quality reconstruction in a problem of perturbation of data within a narrow range of simulation parameters and can be used for input data of large size.
  arXiv  Detail & Related papers  (2021-06-01T08:46:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.