Structured Multi-task Learning for Molecular Property Prediction

• URL: http://arxiv.org/abs/2203.04695v1
• Date: Tue, 22 Feb 2022 20:31:23 GMT
• Title: Structured Multi-task Learning for Molecular Property Prediction
• Authors: Shengchao Liu, Meng Qu, Zuobai Zhang, Huiyu Cai, Jian Tang
• Abstract summary: We study multi-task learning for molecular property prediction in a novel setting, where a relation graph between tasks is available. In the emphlatent space, we model the task representations by applying a state graph neural network (SGNN) on the relation graph. In the emphoutput space, we employ structured prediction with the energy-based model (EBM), which can be efficiently trained through noise-contrastive estimation (NCE) approach.
• Score: 30.77287550003828
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-task learning for molecular property prediction is becoming increasingly important in drug discovery. However, in contrast to other domains, the performance of multi-task learning in drug discovery is still not satisfying as the number of labeled data for each task is too limited, which calls for additional data to complement the data scarcity. In this paper, we study multi-task learning for molecular property prediction in a novel setting, where a relation graph between tasks is available. We first construct a dataset including around 400 tasks as well as a task relation graph. Then to better utilize such relation graph, we propose a method called SGNN-EBM to systematically investigate the structured task modeling from two perspectives. (1) In the \emph{latent} space, we model the task representations by applying a state graph neural network (SGNN) on the relation graph. (2) In the \emph{output} space, we employ structured prediction with the energy-based model (EBM), which can be efficiently trained through noise-contrastive estimation (NCE) approach. Empirical results justify the effectiveness of SGNN-EBM. Code is available on https://github.com/chao1224/SGNN-EBM.

Related papers

• Learning From Graph-Structured Data: Addressing Design Issues and Exploring Practical Applications in Graph Representation Learning [2.492884361833709]
  We present an exhaustive review of the latest advancements in graph representation learning and Graph Neural Networks (GNNs) GNNs, tailored to handle graph-structured data, excel in deriving insights and predictions from intricate relational information. Our work delves into the capabilities of GNNs, examining their foundational designs and their application in addressing real-world challenges.
  arXiv  Detail & Related papers  (2024-11-09T19:10:33Z)
• In-Context Learning of Physical Properties: Few-Shot Adaptation to Out-of-Distribution Molecular Graphs [1.8635507597668244]
  In-context learning allows for performing nontrivial machine learning tasks during inference only. In this work, we address the question: can we leverage in-context learning to predict out-of-distribution materials properties? We employ a compound model in which GPT-2 acts on the output of geometry-aware graph neural networks to adapt in-context information.
  arXiv  Detail & Related papers  (2024-06-03T21:59:21Z)
• Self-Supervised Neuron Segmentation with Multi-Agent Reinforcement Learning [53.00683059396803]
  Mask image model (MIM) has been widely used due to its simplicity and effectiveness in recovering original information from masked images. We propose a decision-based MIM that utilizes reinforcement learning (RL) to automatically search for optimal image masking ratio and masking strategy. Our approach has a significant advantage over alternative self-supervised methods on the task of neuron segmentation.
  arXiv  Detail & Related papers  (2023-10-06T10:40:46Z)
• Diffusion Model is an Effective Planner and Data Synthesizer for Multi-Task Reinforcement Learning [101.66860222415512]
  Multi-Task Diffusion Model (textscMTDiff) is a diffusion-based method that incorporates Transformer backbones and prompt learning for generative planning and data synthesis. For generative planning, we find textscMTDiff outperforms state-of-the-art algorithms across 50 tasks on Meta-World and 8 maps on Maze2D.
  arXiv  Detail & Related papers  (2023-05-29T05:20:38Z)
• GIMLET: A Unified Graph-Text Model for Instruction-Based Molecule Zero-Shot Learning [71.89623260998934]
  This study investigates the feasibility of employing natural language instructions to accomplish molecule-related tasks in a zero-shot setting. Existing molecule-text models perform poorly in this setting due to inadequate treatment of instructions and limited capacity for graphs. We propose GIMLET, which unifies language models for both graph and text data.
  arXiv  Detail & Related papers  (2023-05-28T18:27:59Z)
• Time Associated Meta Learning for Clinical Prediction [78.99422473394029]
  We propose a novel time associated meta learning (TAML) method to make effective predictions at multiple future time points. To address the sparsity problem after task splitting, TAML employs a temporal information sharing strategy to augment the number of positive samples. We demonstrate the effectiveness of TAML on multiple clinical datasets, where it consistently outperforms a range of strong baselines.
  arXiv  Detail & Related papers  (2023-03-05T03:54:54Z)
• Self-supervised Learning for Heterogeneous Graph via Structure Information based on Metapath [9.757299837675204]
  Self-supervised representation learning is a potential approach to tackle this problem. In this paper, we propose a SElfsupervised learning method for heterogeneous graph via Structure Information based on Metapath. In order to predict jump number, SESIM uses data itself to generate labels, avoiding time-consuming manual labeling.
  arXiv  Detail & Related papers  (2022-09-09T10:06:18Z)
• Tyger: Task-Type-Generic Active Learning for Molecular Property Prediction [121.97742787439546]
  How to accurately predict the properties of molecules is an essential problem in AI-driven drug discovery. To reduce annotation cost, deep Active Learning methods are developed to select only the most representative and informative data for annotating. We propose a Task-type-generic active learning framework (termed Tyger) that is able to handle different types of learning tasks in a unified manner.
  arXiv  Detail & Related papers  (2022-05-23T12:56:12Z)
• Graph Representation Learning for Multi-Task Settings: a Meta-Learning Approach [5.629161809575013]
  We propose a novel training strategy for graph representation learning, based on meta-learning. Our method avoids the difficulties arising when learning to perform multiple tasks concurrently. We show that the embeddings produced by a model trained with our method can be used to perform multiple tasks with comparable or, surprisingly, even higher performance than both single-task and multi-task end-to-end models.
  arXiv  Detail & Related papers  (2022-01-10T12:58:46Z)
• Learning an Interpretable Graph Structure in Multi-Task Learning [18.293397644865454]
  We present a novel methodology to jointly perform multi-task learning and infer intrinsic relationship among tasks by an interpretable and sparse graph. Our graph is learned simultaneously with model parameters of each task, thus it reflects the critical relationship among tasks in the specific prediction problem.
  arXiv  Detail & Related papers  (2020-09-11T18:58:14Z)
• Multi-Task Learning for Dense Prediction Tasks: A Survey [87.66280582034838]
  Multi-task learning (MTL) techniques have shown promising results w.r.t. performance, computations and/or memory footprint. We provide a well-rounded view on state-of-the-art deep learning approaches for MTL in computer vision.
  arXiv  Detail & Related papers  (2020-04-28T09:15:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.