Attribution assignment for deep-generative sequence models enables interpretability analysis using positive-only data

• URL: http://arxiv.org/abs/2506.23182v2
• Date: Fri, 11 Jul 2025 14:05:02 GMT
• Title: Attribution assignment for deep-generative sequence models enables interpretability analysis using positive-only data
• Abstract summary: Generative machine learning models offer a powerful framework for therapeutic design by efficiently exploring large spaces of biological sequences enriched for desirable properties.<n>Unlike supervised learning methods, which require both positive and negative labeled data, generative models such as LSTMs can be trained solely on positively labeled sequences.<n>The lack of attribution methods for generative models has hindered the ability to extract interpretable biological insights from such models.<n>We developed Generative Attribution Metric Analysis (GAMA), an attribution method for autoregressive generative models based on Integrated Gradients.
• Score: 3.246305861471588
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Generative machine learning models offer a powerful framework for therapeutic design by efficiently exploring large spaces of biological sequences enriched for desirable properties. Unlike supervised learning methods, which require both positive and negative labeled data, generative models such as LSTMs can be trained solely on positively labeled sequences, for example, high-affinity antibodies. This is particularly advantageous in biological settings where negative data are scarce, unreliable, or biologically ill-defined. However, the lack of attribution methods for generative models has hindered the ability to extract interpretable biological insights from such models. To address this gap, we developed Generative Attribution Metric Analysis (GAMA), an attribution method for autoregressive generative models based on Integrated Gradients. We assessed GAMA using synthetic datasets with known ground truths to characterize its statistical behavior and validate its ability to recover biologically relevant features. We further demonstrated the utility of GAMA by applying it to experimental antibody-antigen binding data. GAMA enables model interpretability and the validation of generative sequence design strategies without the need for negative training data.

Related papers

• Investigating the Impact of Histopathological Foundation Models on Regressive Prediction of Homologous Recombination Deficiency [52.50039435394964]
  We systematically evaluate foundation models for regression-based tasks.<n>We extract patch-level features from whole slide images (WSI) using five state-of-the-art foundation models.<n>Models are trained to predict continuous HRD scores based on these extracted features across breast, endometrial, and lung cancer cohorts.
  arXiv  Detail & Related papers  (2026-01-29T14:06:50Z)
• Hyperbolic Genome Embeddings [0.6656737591902598]
  We develop a novel application of hyperbolic CNNs that exploits the evolutionarily-informed structure of biological systems.<n>Our strategy circumvents the need for explicit phylogenetic mapping while discerning key properties of sequences.<n>Our approach even surpasses state-of-the-art performance on seven GUE benchmark datasets.
  arXiv  Detail & Related papers  (2025-07-29T10:06:17Z)
• Leveraging Diffusion Models for Synthetic Data Augmentation in Protein Subcellular Localization Classification [0.0]
  We implement a class-conditional denoising diffusion probabilistic model (DDPM) to produce label-consistent samples.<n>We explore their integration with real data via two hybrid training strategies: Mix Loss and Mix Representation.<n>Our findings highlight the importance of realistic data generation and robust supervision when incorporating generative augmentation into biomedical image classification.
  arXiv  Detail & Related papers  (2025-05-28T22:58:50Z)
• GENERator: A Long-Context Generative Genomic Foundation Model [66.46537421135996]
  We present GENERator, a generative genomic foundation model featuring a context length of 98k base pairs (bp) and 1.2B parameters.<n>Trained on an expansive dataset comprising 386B bp of DNA, the GENERator demonstrates state-of-the-art performance across both established and newly proposed benchmarks.<n>It also shows significant promise in sequence optimization, particularly through the prompt-responsive generation of enhancer sequences with specific activity profiles.
  arXiv  Detail & Related papers  (2025-02-11T05:39:49Z)
• Multi-Modal and Multi-Attribute Generation of Single Cells with CFGen [76.02070962797794]
  This work introduces CellFlow for Generation (CFGen), a flow-based conditional generative model that preserves the inherent discreteness of single-cell data.<n>CFGen generates whole-genome multi-modal single-cell data reliably, improving the recovery of crucial biological data characteristics.
  arXiv  Detail & Related papers  (2024-07-16T14:05:03Z)
• LLM See, LLM Do: Guiding Data Generation to Target Non-Differentiable Objectives [44.781967004009715]
  We study the impact of passive inheritance of model properties by systematically studying the consequences of synthetic data integration. We find that models are surprisingly sensitive towards certain attributes even when the synthetic data prompts appear "neutral" We propose active inheritance as a term to describe intentionally constraining synthetic data according to a non-differentiable objective.
  arXiv  Detail & Related papers  (2024-07-01T17:26:21Z)
• Seeing Unseen: Discover Novel Biomedical Concepts via Geometry-Constrained Probabilistic Modeling [53.7117640028211]
  We present a geometry-constrained probabilistic modeling treatment to resolve the identified issues. We incorporate a suite of critical geometric properties to impose proper constraints on the layout of constructed embedding space. A spectral graph-theoretic method is devised to estimate the number of potential novel classes.
  arXiv  Detail & Related papers  (2024-03-02T00:56:05Z)
• Improving Biomedical Entity Linking with Retrieval-enhanced Learning [53.24726622142558]
  $k$NN-BioEL provides a BioEL model with the ability to reference similar instances from the entire training corpus as clues for prediction. We show that $k$NN-BioEL outperforms state-of-the-art baselines on several datasets.
  arXiv  Detail & Related papers  (2023-12-15T14:04:23Z)
• Combining propensity score methods with variational autoencoders for generating synthetic data in presence of latent sub-groups [0.0]
  Heterogeneity might be known, e.g., as indicated by sub-groups labels, or might be unknown and reflected only in properties of distributions, such as bimodality or skewness. We investigate how such heterogeneity can be preserved and controlled when obtaining synthetic data from variational autoencoders (VAEs), i.e., a generative deep learning technique.
  arXiv  Detail & Related papers  (2023-12-12T22:49:24Z)
• Differentiable Agent-based Epidemiology [71.81552021144589]
  We introduce GradABM: a scalable, differentiable design for agent-based modeling that is amenable to gradient-based learning with automatic differentiation. GradABM can quickly simulate million-size populations in few seconds on commodity hardware, integrate with deep neural networks and ingest heterogeneous data sources.
  arXiv  Detail & Related papers  (2022-07-20T07:32:02Z)
• Inference of cell dynamics on perturbation data using adjoint sensitivity [4.606583317143614]
  Data-driven dynamic models of cell biology can be used to predict cell response to unseen perturbations. Recent work had demonstrated the derivation of interpretable models with explicit interaction terms. This work aims to extend the range of applicability of this model inference approach to a diversity of biological systems.
  arXiv  Detail & Related papers  (2021-04-13T19:15:56Z)
• Goal-directed Generation of Discrete Structures with Conditional Generative Models [85.51463588099556]
  We introduce a novel approach to directly optimize a reinforcement learning objective, maximizing an expected reward. We test our methodology on two tasks: generating molecules with user-defined properties and identifying short python expressions which evaluate to a given target value.
  arXiv  Detail & Related papers  (2020-10-05T20:03:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.