Flowing Datasets with Wasserstein over Wasserstein Gradient Flows

• URL: http://arxiv.org/abs/2506.07534v1
• Date: Mon, 09 Jun 2025 08:17:35 GMT
• Title: Flowing Datasets with Wasserstein over Wasserstein Gradient Flows
• Authors: Clément Bonet, Christophe Vauthier, Anna Korba,
• Abstract summary: We propose to represent each class by the associated conditional distribution of features, and to model the dataset as a mixture distribution supported on these classes.<n>We endow this space with a metric structure from optimal transport, namely the Wasserstein over Wasserstein (WoW) distance, derive a differential structure on this space, and define WoW gradient flows.<n>We apply our framework to transfer learning and dataset distillation tasks, leveraging our gradient flow construction as well as novel tractable functionals.
• Score: 7.956223802150937
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Many applications in machine learning involve data represented as probability distributions. The emergence of such data requires radically novel techniques to design tractable gradient flows on probability distributions over this type of (infinite-dimensional) objects. For instance, being able to flow labeled datasets is a core task for applications ranging from domain adaptation to transfer learning or dataset distillation. In this setting, we propose to represent each class by the associated conditional distribution of features, and to model the dataset as a mixture distribution supported on these classes (which are themselves probability distributions), meaning that labeled datasets can be seen as probability distributions over probability distributions. We endow this space with a metric structure from optimal transport, namely the Wasserstein over Wasserstein (WoW) distance, derive a differential structure on this space, and define WoW gradient flows. The latter enables to design dynamics over this space that decrease a given objective functional. We apply our framework to transfer learning and dataset distillation tasks, leveraging our gradient flow construction as well as novel tractable functionals that take the form of Maximum Mean Discrepancies with Sliced-Wasserstein based kernels between probability distributions.

Related papers

• Taming Diffusion for Dataset Distillation with High Representativeness [49.3818035378669]
  D3HR is a novel diffusion-based framework to generate distilled datasets with high representativeness.<n>Our experiments demonstrate that D3HR can achieve higher accuracy across different model architectures.
  arXiv  Detail & Related papers  (2025-05-23T22:05:59Z)
• Wasserstein Flow Matching: Generative modeling over families of distributions [13.620905707751747]
  We propose Wasserstein flow matching (WFM), which lifts flow matching onto families of distributions using the Wasserstein geometry.<n> Notably, WFM is the first algorithm capable of generating distributions in high dimensions, whether represented analytically (as Gaussians) or empirically (as point-clouds)
  arXiv  Detail & Related papers  (2024-11-01T15:55:07Z)
• Generative Assignment Flows for Representing and Learning Joint Distributions of Discrete Data [2.6499018693213316]
  We introduce a novel generative model for the representation of joint probability distributions of discrete random variables.<n>The approach uses measure transport by randomized assignment flows on the statistical submanifold of factorizing distributions.
  arXiv  Detail & Related papers  (2024-06-06T21:58:33Z)
• SimPro: A Simple Probabilistic Framework Towards Realistic Long-Tailed Semi-Supervised Learning [49.94607673097326]
  We propose a highly adaptable framework, designated as SimPro, which does not rely on any predefined assumptions about the distribution of unlabeled data. Our framework, grounded in a probabilistic model, innovatively refines the expectation-maximization algorithm. Our method showcases consistent state-of-the-art performance across diverse benchmarks and data distribution scenarios.
  arXiv  Detail & Related papers  (2024-02-21T03:39:04Z)
• Bures-Wasserstein Means of Graphs [60.42414991820453]
  We propose a novel framework for defining a graph mean via embeddings in the space of smooth graph signal distributions. By finding a mean in this embedding space, we can recover a mean graph that preserves structural information. We establish the existence and uniqueness of the novel graph mean, and provide an iterative algorithm for computing it.
  arXiv  Detail & Related papers  (2023-05-31T11:04:53Z)
• Dynamic Flows on Curved Space Generated by Labeled Data [17.621847430986854]
  We propose a gradient flow method to generate new samples close to a dataset of interest. We show that our method can improve the accuracy of classification models in transfer learning settings.
  arXiv  Detail & Related papers  (2023-01-31T19:53:01Z)
• ManiFlow: Implicitly Representing Manifolds with Normalizing Flows [145.9820993054072]
  Normalizing Flows (NFs) are flexible explicit generative models that have been shown to accurately model complex real-world data distributions. We propose an optimization objective that recovers the most likely point on the manifold given a sample from the perturbed distribution. Finally, we focus on 3D point clouds for which we utilize the explicit nature of NFs, i.e. surface normals extracted from the gradient of the log-likelihood and the log-likelihood itself.
  arXiv  Detail & Related papers  (2022-08-18T16:07:59Z)
• VQ-Flows: Vector Quantized Local Normalizing Flows [2.7998963147546148]
  We introduce a novel statistical framework for learning a mixture of local normalizing flows as "chart maps" over a data manifold. Our framework augments the expressivity of recent approaches while preserving the signature property of normalizing flows, that they admit exact density evaluation.
  arXiv  Detail & Related papers  (2022-03-22T09:22:18Z)
• GFlowNet Foundations [66.69854262276391]
  Generative Flow Networks (GFlowNets) have been introduced as a method to sample a diverse set of candidates in an active learning context. We show a number of additional theoretical properties of GFlowNets.
  arXiv  Detail & Related papers  (2021-11-17T17:59:54Z)
• Large-Scale Wasserstein Gradient Flows [84.73670288608025]
  We introduce a scalable scheme to approximate Wasserstein gradient flows. Our approach relies on input neural networks (ICNNs) to discretize the JKO steps. As a result, we can sample from the measure at each step of the gradient diffusion and compute its density.
  arXiv  Detail & Related papers  (2021-06-01T19:21:48Z)
• Dataset Dynamics via Gradient Flows in Probability Space [15.153110906331733]
  We propose a novel framework for dataset transformation, which we cast as optimization over data-generating joint probability distributions. We show that this framework can be used to impose constraints on classification datasets, adapt them for transfer learning, or to re-purpose fixed or black-box models to classify -- with high accuracy -- previously unseen datasets.
  arXiv  Detail & Related papers  (2020-10-24T03:29:22Z)
• Graph Embedding with Data Uncertainty [113.39838145450007]
  spectral-based subspace learning is a common data preprocessing step in many machine learning pipelines. Most subspace learning methods do not take into consideration possible measurement inaccuracies or artifacts that can lead to data with high uncertainty.
  arXiv  Detail & Related papers  (2020-09-01T15:08:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.