Dataset Dynamics via Gradient Flows in Probability Space

• URL: http://arxiv.org/abs/2010.12760v2
• Date: Wed, 16 Jun 2021 16:33:12 GMT
• Title: Dataset Dynamics via Gradient Flows in Probability Space
• Authors: David Alvarez-Melis and Nicol\`o Fusi
• Abstract summary: We propose a novel framework for dataset transformation, which we cast as optimization over data-generating joint probability distributions. We show that this framework can be used to impose constraints on classification datasets, adapt them for transfer learning, or to re-purpose fixed or black-box models to classify -- with high accuracy -- previously unseen datasets.
• Score: 15.153110906331733
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Various machine learning tasks, from generative modeling to domain adaptation, revolve around the concept of dataset transformation and manipulation. While various methods exist for transforming unlabeled datasets, principled methods to do so for labeled (e.g., classification) datasets are missing. In this work, we propose a novel framework for dataset transformation, which we cast as optimization over data-generating joint probability distributions. We approach this class of problems through Wasserstein gradient flows in probability space, and derive practical and efficient particle-based methods for a flexible but well-behaved class of objective functions. Through various experiments, we show that this framework can be used to impose constraints on classification datasets, adapt them for transfer learning, or to re-purpose fixed or black-box models to classify -- with high accuracy -- previously unseen datasets.

Related papers

• Amortized Probabilistic Conditioning for Optimization, Simulation and Inference [20.314865219675056]
  Amortized Conditioning Engine (ACE) A new transformer-based meta-learning model that explicitly represents latent variables of interest. ACE affords conditioning on both observed data and interpretable latent variables, the inclusion of priors at runtime, and outputs predictive distributions for discrete and continuous data and latents.
  arXiv  Detail & Related papers  (2024-10-20T07:22:54Z)
• Personalized Federated Learning via Active Sampling [50.456464838807115]
  This paper proposes a novel method for sequentially identifying similar (or relevant) data generators. Our method evaluates the relevance of a data generator by evaluating the effect of a gradient step using its local dataset. We extend this method to non-parametric models by a suitable generalization of the gradient step to update a hypothesis using the local dataset provided by a data generator.
  arXiv  Detail & Related papers  (2024-09-03T17:12:21Z)
• Distribution-Aware Data Expansion with Diffusion Models [55.979857976023695]
  We propose DistDiff, a training-free data expansion framework based on the distribution-aware diffusion model. DistDiff consistently enhances accuracy across a diverse range of datasets compared to models trained solely on original data.
  arXiv  Detail & Related papers  (2024-03-11T14:07:53Z)
• Joint Distributional Learning via Cramer-Wold Distance [0.7614628596146602]
  We introduce the Cramer-Wold distance regularization, which can be computed in a closed-form, to facilitate joint distributional learning for high-dimensional datasets. We also introduce a two-step learning method to enable flexible prior modeling and improve the alignment between the aggregated posterior and the prior distribution.
  arXiv  Detail & Related papers  (2023-10-25T05:24:23Z)
• Geometrically Aligned Transfer Encoder for Inductive Transfer in Regression Tasks [5.038936775643437]
  We propose a novel transfer technique based on differential geometry, namely the Geometrically Aligned Transfer (GATE) We find a proper diffeomorphism between pairs of tasks to ensure that every arbitrary point maps to a locally flat coordinate in the overlapping region, allowing the transfer of knowledge from the source to the target data. GATE outperforms conventional methods and exhibits stable behavior in both the latent space and extrapolation regions for various molecular graph datasets.
  arXiv  Detail & Related papers  (2023-10-10T07:11:25Z)
• Binary Quantification and Dataset Shift: An Experimental Investigation [54.14283123210872]
  Quantification is the supervised learning task that consists of training predictors of the class prevalence values of sets of unlabelled data. The relationship between quantification and other types of dataset shift remains, by and large, unexplored. We propose a fine-grained taxonomy of types of dataset shift, by establishing protocols for the generation of datasets affected by these types of shift.
  arXiv  Detail & Related papers  (2023-10-06T20:11:27Z)
• A Data-Driven Method for Automated Data Superposition with Applications in Soft Matter Science [0.0]
  We develop a data-driven, non-parametric method for superposing experimental data with arbitrary coordinate transformations. Our method produces interpretable data-driven models that may inform applications such as materials classification, design, and discovery.
  arXiv  Detail & Related papers  (2022-04-20T14:58:04Z)
• Invariance Learning in Deep Neural Networks with Differentiable Laplace Approximations [76.82124752950148]
  We develop a convenient gradient-based method for selecting the data augmentation. We use a differentiable Kronecker-factored Laplace approximation to the marginal likelihood as our objective.
  arXiv  Detail & Related papers  (2022-02-22T02:51:11Z)
• A Lagrangian Duality Approach to Active Learning [119.36233726867992]
  We consider the batch active learning problem, where only a subset of the training data is labeled. We formulate the learning problem using constrained optimization, where each constraint bounds the performance of the model on labeled samples. We show, via numerical experiments, that our proposed approach performs similarly to or better than state-of-the-art active learning methods.
  arXiv  Detail & Related papers  (2022-02-08T19:18:49Z)
• A Trainable Optimal Transport Embedding for Feature Aggregation and its Relationship to Attention [96.77554122595578]
  We introduce a parametrized representation of fixed size, which embeds and then aggregates elements from a given input set according to the optimal transport plan between the set and a trainable reference. Our approach scales to large datasets and allows end-to-end training of the reference, while also providing a simple unsupervised learning mechanism with small computational cost.
  arXiv  Detail & Related papers  (2020-06-22T08:35:58Z)
• Heterogeneous Transfer Learning in Ensemble Clustering [0.0]
  We consider a clustering problem in which "similar" labeled data are available. The method is based on constructing meta-features which describe structural characteristics of data. An experimental study of the method using Monte Carlo modeling has confirmed its efficiency.
  arXiv  Detail & Related papers  (2020-01-20T16:03:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.