Generative Distribution Embeddings

• URL: http://arxiv.org/abs/2505.18150v1
• Date: Fri, 23 May 2025 17:58:57 GMT
• Title: Generative Distribution Embeddings
• Authors: Nic Fishman, Gokul Gowri, Peng Yin, Jonathan Gootenberg, Omar Abudayyeh,
• Abstract summary: We introduce generative distribution embeddings (GDE), a framework that lifts autoencoders to the space of distributions.<n>In GDEs, an encoder acts on sets of samples, and the decoder is replaced by a generator which aims to match the input distribution.<n>We apply GDEs to six key problems in computational biology.
• Score: 1.3252809892089024
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Many real-world problems require reasoning across multiple scales, demanding models which operate not on single data points, but on entire distributions. We introduce generative distribution embeddings (GDE), a framework that lifts autoencoders to the space of distributions. In GDEs, an encoder acts on sets of samples, and the decoder is replaced by a generator which aims to match the input distribution. This framework enables learning representations of distributions by coupling conditional generative models with encoder networks which satisfy a criterion we call distributional invariance. We show that GDEs learn predictive sufficient statistics embedded in the Wasserstein space, such that latent GDE distances approximately recover the $W_2$ distance, and latent interpolation approximately recovers optimal transport trajectories for Gaussian and Gaussian mixture distributions. We systematically benchmark GDEs against existing approaches on synthetic datasets, demonstrating consistently stronger performance. We then apply GDEs to six key problems in computational biology: learning representations of cell populations from lineage-tracing data (150K cells), predicting perturbation effects on single-cell transcriptomes (1M cells), predicting perturbation effects on cellular phenotypes (20M single-cell images), modeling tissue-specific DNA methylation patterns (253M sequences), designing synthetic yeast promoters (34M sequences), and spatiotemporal modeling of viral protein sequences (1M sequences).

Related papers

• Unlasting: Unpaired Single-Cell Multi-Perturbation Estimation by Dual Conditional Diffusion Implicit Bridges [68.98973318553983]
  We propose a framework based on Dual Diffusion Implicit Bridges (DDIB) to learn the mapping between different data distributions.<n>We integrate gene regulatory network (GRN) information to propagate perturbation signals in a biologically meaningful way.<n>We also incorporate a masking mechanism to predict silent genes, improving the quality of generated profiles.
  arXiv  Detail & Related papers  (2025-06-26T09:05:38Z)
• Fine-Tuning Discrete Diffusion Models via Reward Optimization with Applications to DNA and Protein Design [56.957070405026194]
  We propose an algorithm that enables direct backpropagation of rewards through entire trajectories generated by diffusion models.<n>DRAKES can generate sequences that are both natural-like and yield high rewards.
  arXiv  Detail & Related papers  (2024-10-17T15:10:13Z)
• Bellman Diffusion: Generative Modeling as Learning a Linear Operator in the Distribution Space [72.52365911990935]
  We introduce Bellman Diffusion, a novel DGM framework that maintains linearity in MDPs through gradient and scalar field modeling. Our results show that Bellman Diffusion achieves accurate field estimations and is a capable image generator, converging 1.5x faster than the traditional histogram-based baseline in distributional RL tasks.
  arXiv  Detail & Related papers  (2024-10-02T17:53:23Z)
• Scalable Amortized GPLVMs for Single Cell Transcriptomics Data [9.010523724015398]
  Dimensionality reduction is crucial for analyzing large-scale single-cell RNA-seq data. We introduce an improved model, the amortized variational model (BGPLVM) BGPLVM is tailored for single-cell RNA-seq with specialized encoder, kernel, and likelihood designs.
  arXiv  Detail & Related papers  (2024-05-06T21:54:38Z)
• sc-OTGM: Single-Cell Perturbation Modeling by Solving Optimal Mass Transport on the Manifold of Gaussian Mixtures [0.9674145073701153]
  sc-OTGM is an unsupervised model grounded in the inductive bias that the scRNAseq data can be generated. sc-OTGM is effective in cell state classification, aids in the analysis of differential gene expression, and ranks genes for target identification. It also predicts the effects of single-gene perturbations on downstream gene regulation and generates synthetic scRNA-seq data conditioned on specific cell states.
  arXiv  Detail & Related papers  (2024-05-06T06:46:11Z)
• Synthetic location trajectory generation using categorical diffusion models [50.809683239937584]
  Diffusion models (DPMs) have rapidly evolved to be one of the predominant generative models for the simulation of synthetic data. We propose using DPMs for the generation of synthetic individual location trajectories (ILTs) which are sequences of variables representing physical locations visited by individuals.
  arXiv  Detail & Related papers  (2024-02-19T15:57:39Z)
• Gaussian Mixture Solvers for Diffusion Models [84.83349474361204]
  We introduce a novel class of SDE-based solvers called GMS for diffusion models. Our solver outperforms numerous SDE-based solvers in terms of sample quality in image generation and stroke-based synthesis.
  arXiv  Detail & Related papers  (2023-11-02T02:05:38Z)
• Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution [67.9215891673174]
  We propose score entropy as a novel loss that naturally extends score matching to discrete spaces. We test our Score Entropy Discrete Diffusion models on standard language modeling tasks.
  arXiv  Detail & Related papers  (2023-10-25T17:59:12Z)
• Efficient Integrators for Diffusion Generative Models [22.01769257075573]
  Diffusion models suffer from slow sample generation at inference time. We propose two complementary frameworks for accelerating sample generation in pre-trained models. We present a hybrid method that leads to the best-reported performance for diffusion models in augmented spaces.
  arXiv  Detail & Related papers  (2023-10-11T21:04:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.