Lo-Hi: Practical ML Drug Discovery Benchmark

• URL: http://arxiv.org/abs/2310.06399v1
• Date: Tue, 10 Oct 2023 08:06:32 GMT
• Title: Lo-Hi: Practical ML Drug Discovery Benchmark
• Authors: Simon Steshin
• Abstract summary: One of the hopes of drug discovery is to use machine learning models to predict molecular properties. Existing benchmarks for molecular property prediction are unrealistic and are too different from applying the models in practice. We have created a new practical emphLo-Hi benchmark, corresponding to the real drug discovery process.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Finding new drugs is getting harder and harder. One of the hopes of drug discovery is to use machine learning models to predict molecular properties. That is why models for molecular property prediction are being developed and tested on benchmarks such as MoleculeNet. However, existing benchmarks are unrealistic and are too different from applying the models in practice. We have created a new practical \emph{Lo-Hi} benchmark consisting of two tasks: Lead Optimization (Lo) and Hit Identification (Hi), corresponding to the real drug discovery process. For the Hi task, we designed a novel molecular splitting algorithm that solves the Balanced Vertex Minimum $k$-Cut problem. We tested state-of-the-art and classic ML models, revealing which works better under practical settings. We analyzed modern benchmarks and showed that they are unrealistic and overoptimistic. Review: https://openreview.net/forum?id=H2Yb28qGLV Lo-Hi benchmark: https://github.com/SteshinSS/lohi_neurips2023 Lo-Hi splitter library: https://github.com/SteshinSS/lohi_splitter

Related papers

• Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach [70.44265766483633]
  We study a novel language model architecture that is capable of scaling test-time computation by implicitly reasoning in latent space. Our model works by iterating a recurrent block, thereby unrolling to arbitrary depth at test-time. We show that the resulting model can improve its performance on reasoning benchmarks, sometimes dramatically.
  arXiv  Detail & Related papers  (2025-02-07T18:55:02Z)
• Zeroth-Order Adaptive Neuron Alignment Based Pruning without Re-Training [3.195234044113248]
  We propose textscNeuroAL, a emphtop-up algorithm for network pruning. It modifies the block-wise and row-wise sparsity exploiting information from both the dense model and its sparse version. It consistently outperforms the latest state-of-the-art methods in terms of performance-runtime trade-off.
  arXiv  Detail & Related papers  (2024-11-11T15:30:16Z)
• PMLBmini: A Tabular Classification Benchmark Suite for Data-Scarce Applications [2.3700911865675187]
  PMLBmini is a benchmark suite of 44 binary classification datasets with sample sizes $leq$ 500. We use our suite to thoroughly evaluate current automated machine learning (AutoML) frameworks. Our analysis reveals that state-of-the-art AutoML and deep learning approaches often fail to appreciably outperform even a simple logistic regression baseline.
  arXiv  Detail & Related papers  (2024-09-03T06:13:03Z)
• Promises and Pitfalls of Generative Masked Language Modeling: Theoretical Framework and Practical Guidelines [74.42485647685272]
  We focus on Generative Masked Language Models (GMLMs) We train a model to fit conditional probabilities of the data distribution via masking, which are subsequently used as inputs to a Markov Chain to draw samples from the model. We adapt the T5 model for iteratively-refined parallel decoding, achieving 2-3x speedup in machine translation with minimal sacrifice in quality.
  arXiv  Detail & Related papers  (2024-07-22T18:00:00Z)
• Language models scale reliably with over-training and on downstream tasks [121.69867718185125]
  Scaling laws are useful guides for derisking expensive training runs. However, there remain gaps between current studies and how language models are trained. In contrast, scaling laws mostly predict loss on inference, but models are usually compared on downstream task performance.
  arXiv  Detail & Related papers  (2024-03-13T13:54:00Z)
• Augmented Memory: Capitalizing on Experience Replay to Accelerate De Novo Molecular Design [0.0]
  Molecular generative models should learn to satisfy a desired objective under minimal oracle evaluations. We propose a novel algorithm called Augmented Memory that combines data augmentation with experience replay. We show that scores obtained from oracle calls can be reused to update the model multiple times.
  arXiv  Detail & Related papers  (2023-05-10T14:00:50Z)
• nanoLM: an Affordable LLM Pre-training Benchmark via Accurate Loss Prediction across Scales [65.01417261415833]
  We present an approach to predict the pre-training loss based on our observations that Maximal Update Parametrization (muP) enables accurate fitting of scaling laws. With around 14% of the one-time pre-training cost, we can accurately forecast the loss for models up to 52B. Our goal with nanoLM is to empower researchers with limited resources to reach meaningful conclusions on large models.
  arXiv  Detail & Related papers  (2023-04-14T00:45:01Z)
• MolGraph: a Python package for the implementation of molecular graphs and graph neural networks with TensorFlow and Keras [51.92255321684027]
  MolGraph is a graph neural network (GNN) package for molecular machine learning (ML) MolGraph implements a chemistry module to accommodate the generation of small molecular graphs, which can be passed to a GNN algorithm to solve a molecular ML problem. GNNs proved useful for molecular identification and improved interpretability of chromatographic retention time data.
  arXiv  Detail & Related papers  (2022-08-21T18:37:41Z)
• MIMOSA: Multi-constraint Molecule Sampling for Molecule Optimization [51.00815310242277]
  generative models and reinforcement learning approaches made initial success, but still face difficulties in simultaneously optimizing multiple drug properties. We propose the MultI-constraint MOlecule SAmpling (MIMOSA) approach, a sampling framework to use input molecule as an initial guess and sample molecules from the target distribution.
  arXiv  Detail & Related papers  (2020-10-05T20:18:42Z)
• Continuous Optimization Benchmarks by Simulation [0.0]
  Benchmark experiments are required to test, compare, tune, and understand optimization algorithms. Data from previous evaluations can be used to train surrogate models which are then used for benchmarking. We show that the spectral simulation method enables simulation for continuous optimization problems.
  arXiv  Detail & Related papers  (2020-08-14T08:50:57Z)
• We Should at Least Be Able to Design Molecules That Dock Well [5.751280593108197]
  We propose a benchmark based on docking, a popular computational method for assessing molecule binding to a protein. We observe that popular graph-based generative models fail to generate molecules with a high docking score when trained using a realistically sized training set. We propose a simplified version of the benchmark based on a simpler scoring function, and show that the tested models are able to partially solve it.
  arXiv  Detail & Related papers  (2020-06-20T16:40:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.