GenEM: Physics-Informed Generative Cryo-Electron Microscopy
- URL: http://arxiv.org/abs/2312.02235v1
- Date: Mon, 4 Dec 2023 07:52:56 GMT
- Title: GenEM: Physics-Informed Generative Cryo-Electron Microscopy
- Authors: Jiakai Zhang, Qihe Chen, Yan Zeng, Wenyuan Gao, Xuming He, Zhijie Liu,
Jingyi Yu
- Abstract summary: We introduce physics-informed generative cryo-electron microscopy (GenEM)
GenEM integrates physical-based cryo-EM simulation with a generative unpaired noise translation to generate realistic noises.
Experiments show that GenEM is capable of generating realistic cryo-EM images.
- Score: 40.51069961987814
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: In the past decade, deep conditional generative models have revolutionized
the generation of realistic images, extending their application from
entertainment to scientific domains. Single-particle cryo-electron microscopy
(cryo-EM) is crucial in resolving near-atomic resolution 3D structures of
proteins, such as the SARS-COV-2 spike protein. To achieve high-resolution
reconstruction, AI models for particle picking and pose estimation have been
adopted. However, their performance is still limited as they lack high-quality
annotated datasets. To address this, we introduce physics-informed generative
cryo-electron microscopy (GenEM), which for the first time integrates
physical-based cryo-EM simulation with a generative unpaired noise translation
to generate physically correct synthetic cryo-EM datasets with realistic
noises. Initially, GenEM simulates the cryo-EM imaging process based on a
virtual specimen. To generate realistic noises, we leverage an unpaired noise
translation via contrastive learning with a novel mask-guided sampling scheme.
Extensive experiments show that GenEM is capable of generating realistic
cryo-EM images. The generated dataset can further enhance particle picking and
pose estimation models, eventually improving the reconstruction resolution. We
will release our code and annotated synthetic datasets.
Related papers
- Physics-Driven Turbulence Image Restoration with Stochastic Refinement [80.79900297089176]
Image distortion by atmospheric turbulence is a critical problem in long-range optical imaging systems.
Fast and physics-grounded simulation tools have been introduced to help the deep-learning models adapt to real-world turbulence conditions.
This paper proposes the Physics-integrated Restoration Network (PiRN) to help the network to disentangle theity from the degradation and the underlying image.
arXiv Detail & Related papers (2023-07-20T05:49:21Z) - CryoFormer: Continuous Heterogeneous Cryo-EM Reconstruction using
Transformer-based Neural Representations [49.49939711956354]
Cryo-electron microscopy (cryo-EM) allows for the high-resolution reconstruction of 3D structures of proteins and other biomolecules.
It is still challenging to reconstruct the continuous motions of 3D structures from noisy and randomly oriented 2D cryo-EM images.
We propose CryoFormer, a new approach for continuous heterogeneous cryo-EM reconstruction.
arXiv Detail & Related papers (2023-03-28T18:59:17Z) - Pixelated Reconstruction of Foreground Density and Background Surface
Brightness in Gravitational Lensing Systems using Recurrent Inference
Machines [116.33694183176617]
We use a neural network based on the Recurrent Inference Machine to reconstruct an undistorted image of the background source and the lens mass density distribution as pixelated maps.
When compared to more traditional parametric models, the proposed method is significantly more expressive and can reconstruct complex mass distributions.
arXiv Detail & Related papers (2023-01-10T19:00:12Z) - FSID: Fully Synthetic Image Denoising via Procedural Scene Generation [12.277286575812441]
We develop a procedural synthetic data generation pipeline and dataset tailored to low-level vision tasks.
Our Unreal engine-based synthetic data pipeline populates large scenes algorithmically with a combination of random 3D objects, materials, and geometric transformations.
We then trained and validated a CNN-based denoising model, and demonstrated that the model trained on this synthetic data alone can achieve competitive denoising results.
arXiv Detail & Related papers (2022-12-07T21:21:55Z) - Amortized Inference for Heterogeneous Reconstruction in Cryo-EM [36.911133113707045]
cryo-electron microscopy (cryo-EM) provides insights into the dynamics of proteins and other building blocks of life.
The algorithmic challenge of jointly estimating the poses, 3D structure, and conformational heterogeneity of a biomolecule remains unsolved.
Our method, cryoFIRE, performs ab initio heterogeneous reconstruction with unknown poses in an amortized framework.
We show that our method can provide one order of magnitude speedup on datasets containing millions of images without any loss of accuracy.
arXiv Detail & Related papers (2022-10-13T22:06:38Z) - Differentiable Electron Microscopy Simulation: Methods and Applications
for Visualization [40.8023670606058]
We propose a new microscopy simulation system that can depict atomistic models in a micrograph visual style.
The system is scalable, able to represent simulation of electron microscopy of tens of viral particles and synthesizes the image faster than previous methods.
arXiv Detail & Related papers (2022-05-08T12:39:04Z) - CryoAI: Amortized Inference of Poses for Ab Initio Reconstruction of 3D
Molecular Volumes from Real Cryo-EM Images [30.738209997049395]
We introduce cryoAI, an ab initio reconstruction algorithm for homogeneous conformations that uses gradient-based optimization of particle poses and the electron scattering potential from single-particle cryo-EM data.
CryoAI achieves results on par with state-of-the-art cryo-EM solvers for both simulated and experimental data, one order of magnitude faster for large datasets and with significantly lower memory requirements than existing methods.
arXiv Detail & Related papers (2022-03-15T17:58:03Z) - Data-driven generation of plausible tissue geometries for realistic
photoacoustic image synthesis [53.65837038435433]
Photoacoustic tomography (PAT) has the potential to recover morphological and functional tissue properties.
We propose a novel approach to PAT data simulation, which we refer to as "learning to simulate"
We leverage the concept of Generative Adversarial Networks (GANs) trained on semantically annotated medical imaging data to generate plausible tissue geometries.
arXiv Detail & Related papers (2021-03-29T11:30:18Z) - Unlimited Resolution Image Generation with R2D2-GANs [69.90258455164513]
We present a novel simulation technique for generating high quality images of any predefined resolution.
This method can be used to synthesize sonar scans of size equivalent to those collected during a full-length mission.
The data produced is continuous, realistically-looking, and can also be generated at least two times faster than the real speed of acquisition.
arXiv Detail & Related papers (2020-03-02T17:49:32Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.