Fractional Denoising for 3D Molecular Pre-training

• URL: http://arxiv.org/abs/2307.10683v3
• Date: Tue, 27 Feb 2024 02:26:27 GMT
• Title: Fractional Denoising for 3D Molecular Pre-training
• Authors: Shikun Feng and Yuyan Ni and Yanyan Lan and Zhi-Ming Ma and Wei-Ying Ma
• Abstract summary: Coordinate denoising is a promising 3D molecular pre-training method, which has achieved remarkable performance in various downstream drug discovery tasks. There are two challenges for coordinate denoising to learn an effective force field, i.e. low coverage samples and isotropic force field. We propose a novel hybrid noise strategy, including noises on both dihedral angel and coordinate.
• Score: 29.671249096191726
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Coordinate denoising is a promising 3D molecular pre-training method, which has achieved remarkable performance in various downstream drug discovery tasks. Theoretically, the objective is equivalent to learning the force field, which is revealed helpful for downstream tasks. Nevertheless, there are two challenges for coordinate denoising to learn an effective force field, i.e. low coverage samples and isotropic force field. The underlying reason is that molecular distributions assumed by existing denoising methods fail to capture the anisotropic characteristic of molecules. To tackle these challenges, we propose a novel hybrid noise strategy, including noises on both dihedral angel and coordinate. However, denoising such hybrid noise in a traditional way is no more equivalent to learning the force field. Through theoretical deductions, we find that the problem is caused by the dependency of the input conformation for covariance. To this end, we propose to decouple the two types of noise and design a novel fractional denoising method (Frad), which only denoises the latter coordinate part. In this way, Frad enjoys both the merits of sampling more low-energy structures and the force field equivalence. Extensive experiments show the effectiveness of Frad in molecular representation, with a new state-of-the-art on 9 out of 12 tasks of QM9 and on 7 out of 8 targets of MD17.

Related papers

• Noise Conditional Variational Score Distillation [60.38982038894823]
  Noise Conditional Variational Score Distillation (NCVSD) is a novel method for distilling pretrained diffusion models into generative denoisers.<n>By integrating this insight into the Variational Score Distillation framework, we enable scalable learning of generative denoisers.
  arXiv  Detail & Related papers  (2025-06-11T06:01:39Z)
• Divide and Conquer: Heterogeneous Noise Integration for Diffusion-based Adversarial Purification [75.09791002021947]
  Existing purification methods aim to disrupt adversarial perturbations by introducing a certain amount of noise through a forward diffusion process, followed by a reverse process to recover clean examples. This approach is fundamentally flawed as the uniform operation of the forward process compromises normal pixels while attempting to combat adversarial perturbations. We propose a heterogeneous purification strategy grounded in the interpretability of neural networks. Our method decisively applies higher-intensity noise to specific pixels that the target model focuses on while the remaining pixels are subjected to only low-intensity noise.
  arXiv  Detail & Related papers  (2025-03-03T11:00:25Z)
• Pre-training with Fractional Denoising to Enhance Molecular Property Prediction [26.93248595345132]
  We introduce a molecular pre-training framework called fractional denoising (Frad), which decouples noise design from the constraints imposed by force learning equivalence. Experiments demonstrate that our framework consistently outperforms existing methods, establishing state-of-the-art results across force prediction, quantum chemical properties, and binding affinity tasks.
  arXiv  Detail & Related papers  (2024-07-14T11:09:42Z)
• NoiseDiffusion: Correcting Noise for Image Interpolation with Diffusion Models beyond Spherical Linear Interpolation [86.7260950382448]
  We propose a novel approach to correct noise for image validity, NoiseDiffusion. NoiseDiffusion performs within the noisy image space and injects raw images into these noisy counterparts to address the challenge of information loss.
  arXiv  Detail & Related papers  (2024-03-13T12:32:25Z)
• GeneOH Diffusion: Towards Generalizable Hand-Object Interaction Denoising via Denoising Diffusion [27.252526086004956]
  Given an erroneous interaction sequence, the objective is to refine the incorrect hand trajectory to remove interaction artifacts for a perceptually realistic sequence. This challenge involves intricate interaction noise, including unnatural hand poses and incorrect hand-object relations. We tackle those challenges through a novel approach, GeneOH Diffusion, incorporating two key designs.
  arXiv  Detail & Related papers  (2024-02-22T18:59:21Z)
• Sliced Denoising: A Physics-Informed Molecular Pre-Training Method [29.671249096191726]
  This paper proposes a new method for molecular pre-training, called sliced denoising (SliDe) SliDe uses a novel noise strategy that perturbs bond lengths, angles, and torsion angles to achieve better sampling over conformations. It shows a 42% improvement in the accuracy of estimated force fields compared to current state-of-the-art denoising methods.
  arXiv  Detail & Related papers  (2023-11-03T07:58:05Z)
• DeNoising-MOT: Towards Multiple Object Tracking with Severe Occlusions [52.63323657077447]
  We propose DNMOT, an end-to-end trainable DeNoising Transformer for multiple object tracking. Specifically, we augment the trajectory with noises during training and make our model learn the denoising process in an encoder-decoder architecture. We conduct extensive experiments on the MOT17, MOT20, and DanceTrack datasets, and the experimental results show that our method outperforms previous state-of-the-art methods by a clear margin.
  arXiv  Detail & Related papers  (2023-09-09T04:40:01Z)
• CamoDiffusion: Camouflaged Object Detection via Conditional Diffusion Models [72.93652777646233]
  Camouflaged Object Detection (COD) is a challenging task in computer vision due to the high similarity between camouflaged objects and their surroundings. We propose a new paradigm that treats COD as a conditional mask-generation task leveraging diffusion models. Our method, dubbed CamoDiffusion, employs the denoising process of diffusion models to iteratively reduce the noise of the mask.
  arXiv  Detail & Related papers  (2023-05-29T07:49:44Z)
• DiffusionAD: Norm-guided One-step Denoising Diffusion for Anomaly Detection [80.20339155618612]
  DiffusionAD is a novel anomaly detection pipeline comprising a reconstruction sub-network and a segmentation sub-network.<n>A rapid one-step denoising paradigm achieves hundreds of times acceleration while preserving comparable reconstruction quality.<n>Considering the diversity in the manifestation of anomalies, we propose a norm-guided paradigm to integrate the benefits of multiple noise scales.
  arXiv  Detail & Related papers  (2023-03-15T16:14:06Z)
• Molecular Geometry Pretraining with SE(3)-Invariant Denoising Distance Matching [36.92992265307818]
  The power of pretraining on 3D geometric structures has been less explored. We propose a 3D coordinate denoising pretraining framework to model such an energy landscape. Our experiments confirm the effectiveness and robustness of our proposed method.
  arXiv  Detail & Related papers  (2022-06-27T19:30:53Z)
• Pre-training via Denoising for Molecular Property Prediction [53.409242538744444]
  We describe a pre-training technique that utilizes large datasets of 3D molecular structures at equilibrium. Inspired by recent advances in noise regularization, our pre-training objective is based on denoising.
  arXiv  Detail & Related papers  (2022-05-31T22:28:34Z)
• Combating Noise: Semi-supervised Learning by Region Uncertainty Quantification [55.23467274564417]
  Current methods are easily distracted by noisy regions generated by pseudo labels. We propose noise-resistant semi-supervised learning by quantifying the region uncertainty. Experiments on both PASCAL VOC and MS COCO demonstrate the extraordinary performance of our method.
  arXiv  Detail & Related papers  (2021-11-01T13:23:42Z)
• Mixed Noise Removal with Pareto Prior [31.306541517685137]
  Denoising images contaminated by the mixture of additive white Gaussian noise (AWGN) and noise impulse (IN) is an essential but challenging problem. Existing methods target to compensate the effects of IN by introducing a weighting matrix. We propose an accurate and robust weight estimator for mixed noise removal based on the priori of the weighting matrix.
  arXiv  Detail & Related papers  (2020-08-27T06:35:15Z)
• Simultaneous Denoising and Dereverberation Using Deep Embedding Features [64.58693911070228]
  We propose a joint training method for simultaneous speech denoising and dereverberation using deep embedding features. At the denoising stage, the DC network is leveraged to extract noise-free deep embedding features. At the dereverberation stage, instead of using the unsupervised K-means clustering algorithm, another neural network is utilized to estimate the anechoic speech.
  arXiv  Detail & Related papers  (2020-04-06T06:34:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.