Unified Molecule Generation and Property Prediction

• URL: http://arxiv.org/abs/2504.16559v1
• Date: Wed, 23 Apr 2025 09:36:46 GMT
• Title: Unified Molecule Generation and Property Prediction
• Authors: Adam Izdebski, Jan Olszewski, Pankhil Gawade, Krzysztof Koras, Serra Korkmaz, Valentin Rauscher, Jakub M. Tomczak, Ewa Szczurek,
• Abstract summary: Hyformer is a transformer-based joint model that blends the generative and predictive functionalities.<n>We show that Hyformer rivals other joint models, as well as state-of-the-art molecule generation and property prediction models.
• Score: 6.865957689890204
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modeling the joint distribution of the data samples and their properties allows to construct a single model for both data generation and property prediction, with synergistic capabilities reaching beyond purely generative or predictive models. However, training joint models presents daunting architectural and optimization challenges. Here, we propose Hyformer, a transformer-based joint model that successfully blends the generative and predictive functionalities, using an alternating attention mask together with a unified pre-training scheme. We show that Hyformer rivals other joint models, as well as state-of-the-art molecule generation and property prediction models. Additionally, we show the benefits of joint modeling in downstream tasks of molecular representation learning, hit identification and antimicrobial peptide design.

Related papers

• Disjoint Generative Models [3.1318583335754333]
  We propose a new framework for generating cross-sectional synthetic datasets via disjoint generative models.<n>In this paradigm, a dataset is partitioned into disjoint subsets that are supplied to separate instances of generative models.<n>Results are then combined post hoc by a joining operation that works in the absence of common variables/identifiers.
  arXiv  Detail & Related papers  (2025-07-25T22:38:06Z)
• UniGenX: Unified Generation of Sequence and Structure with Autoregressive Diffusion [61.690978792873196]
  Existing approaches rely on either autoregressive sequence models or diffusion models.<n>We propose UniGenX, a unified framework that combines autoregressive next-token prediction with conditional diffusion models.<n>We validate the effectiveness of UniGenX on material and small molecule generation tasks.
  arXiv  Detail & Related papers  (2025-03-09T16:43:07Z)
• A Generalized Theory of Mixup for Structure-Preserving Synthetic Data [2.184775414778289]
  We show that mixup can distort key statistical properties such as variance, potentially leading to unintended consequences in data synthesis.<n>We propose a novel mixup method that incorporates a generalized and flexible weighting scheme, better preserving the original data's structure.<n> Numerical experiments confirm that the new approach not only preserves the statistical characteristics of the original data but also sustains model performance across repeated synthesis.
  arXiv  Detail & Related papers  (2025-03-03T14:28:50Z)
• Bridging Model-Based Optimization and Generative Modeling via Conservative Fine-Tuning of Diffusion Models [54.132297393662654]
  We introduce a hybrid method that fine-tunes cutting-edge diffusion models by optimizing reward models through RL. We demonstrate the capability of our approach to outperform the best designs in offline data, leveraging the extrapolation capabilities of reward models.
  arXiv  Detail & Related papers  (2024-05-30T03:57:29Z)
• InterHandGen: Two-Hand Interaction Generation via Cascaded Reverse Diffusion [53.90516061351706]
  We present InterHandGen, a novel framework that learns the generative prior of two-hand interaction. For sampling, we combine anti-penetration and synthesis-free guidance to enable plausible generation. Our method significantly outperforms baseline generative models in terms of plausibility and diversity.
  arXiv  Detail & Related papers  (2024-03-26T06:35:55Z)
• HyPer-EP: Meta-Learning Hybrid Personalized Models for Cardiac Electrophysiology [7.230055455268642]
  We present a novel hybrid modeling framework to describe a personalized cardiac digital twin. We then present a novel meta-learning framework to enable the separate identification of both the physics-based and neural components.
  arXiv  Detail & Related papers  (2024-03-15T02:30:00Z)
• MGE: A Training-Free and Efficient Model Generation and Enhancement Scheme [10.48591131837771]
  This paper proposes a Training-Free and Efficient Model Generation and Enhancement Scheme (MGE) It considers two aspects during the model generation process: the distribution of model parameters and model performance. Experiments result shows that generated models are comparable to models obtained through normal training, and even superior in some cases.
  arXiv  Detail & Related papers  (2024-02-27T13:12:00Z)
• Co-modeling the Sequential and Graphical Routes for Peptide Representation Learning [67.66393016797181]
  We propose a peptide co-modeling method, RepCon, to enhance the mutual information of representations from decoupled sequential and graphical end-to-end models. RepCon learns to enhance the consistency of representations between positive sample pairs and to repel representations between negative pairs. Our results demonstrate the superiority of the co-modeling approach over independent modeling, as well as the superiority of RepCon over other methods under the co-modeling framework.
  arXiv  Detail & Related papers  (2023-10-04T16:58:25Z)
• Molecule Design by Latent Space Energy-Based Modeling and Gradual Distribution Shifting [53.44684898432997]
  Generation of molecules with desired chemical and biological properties is critical for drug discovery. We propose a probabilistic generative model to capture the joint distribution of molecules and their properties. Our method achieves very strong performances on various molecule design tasks.
  arXiv  Detail & Related papers  (2023-06-09T03:04:21Z)
• Learning Data Representations with Joint Diffusion Models [20.25147743706431]
  Joint machine learning models that allow synthesizing and classifying data often offer uneven performance between those tasks or are unstable to train. We extend the vanilla diffusion model with a classifier that allows for stable joint end-to-end training with shared parameterization between those objectives. The resulting joint diffusion model outperforms recent state-of-the-art hybrid methods in terms of both classification and generation quality on all evaluated benchmarks.
  arXiv  Detail & Related papers  (2023-01-31T13:29:19Z)
• Conditional Generative Models for Simulation of EMG During Naturalistic Movements [45.698312905115955]
  We present a conditional generative neural network trained adversarially to generate motor unit activation potential waveforms. We demonstrate the ability of such a model to predictively interpolate between a much smaller number of numerical model's outputs with a high accuracy.
  arXiv  Detail & Related papers  (2022-11-03T14:49:02Z)
• Hybrid Feature- and Similarity-Based Models for Prediction and Interpretation using Large-Scale Observational Data [0.0]
  We propose a hybrid feature- and similarity-based model for supervised learning. The proposed hybrid model is fit by convex optimization with a sparsity-inducing penalty on the kernel portion. We compared our models to solely feature- and similarity-based approaches using synthetic data and using EHR data to predict risk of loneliness or social isolation.
  arXiv  Detail & Related papers  (2022-04-12T20:37:03Z)
• Test-time Collective Prediction [73.74982509510961]
  Multiple parties in machine learning want to jointly make predictions on future test points. Agents wish to benefit from the collective expertise of the full set of agents, but may not be willing to release their data or model parameters. We explore a decentralized mechanism to make collective predictions at test time, leveraging each agent's pre-trained model.
  arXiv  Detail & Related papers  (2021-06-22T18:29:58Z)
• Extending Models Via Gradient Boosting: An Application to Mendelian Models [1.9573380763700712]
  We propose a general approach to model improvement: we combine gradient boosting with any previously developed model to improve model performance. We show that integration of gradient boosting with an existing Mendelian model can produce an improved model that outperforms both that model and the model built using gradient boosting alone.
  arXiv  Detail & Related papers  (2021-05-13T21:21:05Z)
• Learning Neural Generative Dynamics for Molecular Conformation Generation [89.03173504444415]
  We study how to generate molecule conformations (textiti.e., 3D structures) from a molecular graph. We propose a novel probabilistic framework to generate valid and diverse conformations given a molecular graph.
  arXiv  Detail & Related papers  (2021-02-20T03:17:58Z)
• Conditional Generative Models for Counterfactual Explanations [0.0]
  We propose a general framework to generate sparse, in-distribution counterfactual model explanations. The framework is flexible with respect to the type of generative model used as well as the task of the underlying predictive model.
  arXiv  Detail & Related papers  (2021-01-25T14:31:13Z)
• Improving the Reconstruction of Disentangled Representation Learners via Multi-Stage Modeling [54.94763543386523]
  Current autoencoder-based disentangled representation learning methods achieve disentanglement by penalizing the ( aggregate) posterior to encourage statistical independence of the latent factors. We present a novel multi-stage modeling approach where the disentangled factors are first learned using a penalty-based disentangled representation learning method. Then, the low-quality reconstruction is improved with another deep generative model that is trained to model the missing correlated latent variables.
  arXiv  Detail & Related papers  (2020-10-25T18:51:15Z)
• Goal-directed Generation of Discrete Structures with Conditional Generative Models [85.51463588099556]
  We introduce a novel approach to directly optimize a reinforcement learning objective, maximizing an expected reward. We test our methodology on two tasks: generating molecules with user-defined properties and identifying short python expressions which evaluate to a given target value.
  arXiv  Detail & Related papers  (2020-10-05T20:03:13Z)
• Generalization Properties of Optimal Transport GANs with Latent Distribution Learning [52.25145141639159]
  We study how the interplay between the latent distribution and the complexity of the pushforward map affects performance. Motivated by our analysis, we advocate learning the latent distribution as well as the pushforward map within the GAN paradigm.
  arXiv  Detail & Related papers  (2020-07-29T07:31:33Z)
• Hybrid modeling: Applications in real-time diagnosis [64.5040763067757]
  We outline a novel hybrid modeling approach that combines machine learning inspired models and physics-based models. We are using such models for real-time diagnosis applications.
  arXiv  Detail & Related papers  (2020-03-04T00:44:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.