Fractional Denoising for 3D Molecular Pre-training
- URL: http://arxiv.org/abs/2307.10683v3
- Date: Tue, 27 Feb 2024 02:26:27 GMT
- Title: Fractional Denoising for 3D Molecular Pre-training
- Authors: Shikun Feng and Yuyan Ni and Yanyan Lan and Zhi-Ming Ma and Wei-Ying
Ma
- Abstract summary: Coordinate denoising is a promising 3D molecular pre-training method, which has achieved remarkable performance in various downstream drug discovery tasks.
There are two challenges for coordinate denoising to learn an effective force field, i.e. low coverage samples and isotropic force field.
We propose a novel hybrid noise strategy, including noises on both dihedral angel and coordinate.
- Score: 29.671249096191726
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: Coordinate denoising is a promising 3D molecular pre-training method, which
has achieved remarkable performance in various downstream drug discovery tasks.
Theoretically, the objective is equivalent to learning the force field, which
is revealed helpful for downstream tasks. Nevertheless, there are two
challenges for coordinate denoising to learn an effective force field, i.e. low
coverage samples and isotropic force field. The underlying reason is that
molecular distributions assumed by existing denoising methods fail to capture
the anisotropic characteristic of molecules. To tackle these challenges, we
propose a novel hybrid noise strategy, including noises on both dihedral angel
and coordinate. However, denoising such hybrid noise in a traditional way is no
more equivalent to learning the force field. Through theoretical deductions, we
find that the problem is caused by the dependency of the input conformation for
covariance. To this end, we propose to decouple the two types of noise and
design a novel fractional denoising method (Frad), which only denoises the
latter coordinate part. In this way, Frad enjoys both the merits of sampling
more low-energy structures and the force field equivalence. Extensive
experiments show the effectiveness of Frad in molecular representation, with a
new state-of-the-art on 9 out of 12 tasks of QM9 and on 7 out of 8 targets of
MD17.
Related papers
- Pre-training with Fractional Denoising to Enhance Molecular Property Prediction [26.93248595345132]
We introduce a molecular pre-training framework called fractional denoising (Frad), which decouples noise design from the constraints imposed by force learning equivalence.
Experiments demonstrate that our framework consistently outperforms existing methods, establishing state-of-the-art results across force prediction, quantum chemical properties, and binding affinity tasks.
arXiv Detail & Related papers (2024-07-14T11:09:42Z) - Immiscible Diffusion: Accelerating Diffusion Training with Noise Assignment [56.609042046176555]
Current methods diffuse each image across the entire noise space, resulting in a mixture of all images at every point in the noise layer.
We propose Immiscible Diffusion, a simple and effective method to improve the random mixture of noise-data mapping.
Our approach is remarkably simple, requiring only one line of code to restrict the diffuse-able area for each image.
arXiv Detail & Related papers (2024-06-18T06:20:42Z) - NoiseDiffusion: Correcting Noise for Image Interpolation with Diffusion Models beyond Spherical Linear Interpolation [86.7260950382448]
We propose a novel approach to correct noise for image validity, NoiseDiffusion.
NoiseDiffusion performs within the noisy image space and injects raw images into these noisy counterparts to address the challenge of information loss.
arXiv Detail & Related papers (2024-03-13T12:32:25Z) - GeneOH Diffusion: Towards Generalizable Hand-Object Interaction
Denoising via Denoising Diffusion [27.252526086004956]
Given an erroneous interaction sequence, the objective is to refine the incorrect hand trajectory to remove interaction artifacts for a perceptually realistic sequence.
This challenge involves intricate interaction noise, including unnatural hand poses and incorrect hand-object relations.
We tackle those challenges through a novel approach, GeneOH Diffusion, incorporating two key designs.
arXiv Detail & Related papers (2024-02-22T18:59:21Z) - Sliced Denoising: A Physics-Informed Molecular Pre-Training Method [29.671249096191726]
This paper proposes a new method for molecular pre-training, called sliced denoising (SliDe)
SliDe uses a novel noise strategy that perturbs bond lengths, angles, and torsion angles to achieve better sampling over conformations.
It shows a 42% improvement in the accuracy of estimated force fields compared to current state-of-the-art denoising methods.
arXiv Detail & Related papers (2023-11-03T07:58:05Z) - DeNoising-MOT: Towards Multiple Object Tracking with Severe Occlusions [52.63323657077447]
We propose DNMOT, an end-to-end trainable DeNoising Transformer for multiple object tracking.
Specifically, we augment the trajectory with noises during training and make our model learn the denoising process in an encoder-decoder architecture.
We conduct extensive experiments on the MOT17, MOT20, and DanceTrack datasets, and the experimental results show that our method outperforms previous state-of-the-art methods by a clear margin.
arXiv Detail & Related papers (2023-09-09T04:40:01Z) - CamoDiffusion: Camouflaged Object Detection via Conditional Diffusion
Models [72.93652777646233]
Camouflaged Object Detection (COD) is a challenging task in computer vision due to the high similarity between camouflaged objects and their surroundings.
We propose a new paradigm that treats COD as a conditional mask-generation task leveraging diffusion models.
Our method, dubbed CamoDiffusion, employs the denoising process of diffusion models to iteratively reduce the noise of the mask.
arXiv Detail & Related papers (2023-05-29T07:49:44Z) - Molecular Geometry Pretraining with SE(3)-Invariant Denoising Distance
Matching [36.92992265307818]
The power of pretraining on 3D geometric structures has been less explored.
We propose a 3D coordinate denoising pretraining framework to model such an energy landscape.
Our experiments confirm the effectiveness and robustness of our proposed method.
arXiv Detail & Related papers (2022-06-27T19:30:53Z) - Pre-training via Denoising for Molecular Property Prediction [53.409242538744444]
We describe a pre-training technique that utilizes large datasets of 3D molecular structures at equilibrium.
Inspired by recent advances in noise regularization, our pre-training objective is based on denoising.
arXiv Detail & Related papers (2022-05-31T22:28:34Z) - Mixed Noise Removal with Pareto Prior [31.306541517685137]
Denoising images contaminated by the mixture of additive white Gaussian noise (AWGN) and noise impulse (IN) is an essential but challenging problem.
Existing methods target to compensate the effects of IN by introducing a weighting matrix.
We propose an accurate and robust weight estimator for mixed noise removal based on the priori of the weighting matrix.
arXiv Detail & Related papers (2020-08-27T06:35:15Z) - Simultaneous Denoising and Dereverberation Using Deep Embedding Features [64.58693911070228]
We propose a joint training method for simultaneous speech denoising and dereverberation using deep embedding features.
At the denoising stage, the DC network is leveraged to extract noise-free deep embedding features.
At the dereverberation stage, instead of using the unsupervised K-means clustering algorithm, another neural network is utilized to estimate the anechoic speech.
arXiv Detail & Related papers (2020-04-06T06:34:01Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.