Dataset Dynamics via Gradient Flows in Probability Space
- URL: http://arxiv.org/abs/2010.12760v2
- Date: Wed, 16 Jun 2021 16:33:12 GMT
- Title: Dataset Dynamics via Gradient Flows in Probability Space
- Authors: David Alvarez-Melis and Nicol\`o Fusi
- Abstract summary: We propose a novel framework for dataset transformation, which we cast as optimization over data-generating joint probability distributions.
We show that this framework can be used to impose constraints on classification datasets, adapt them for transfer learning, or to re-purpose fixed or black-box models to classify -- with high accuracy -- previously unseen datasets.
- Score: 15.153110906331733
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Various machine learning tasks, from generative modeling to domain
adaptation, revolve around the concept of dataset transformation and
manipulation. While various methods exist for transforming unlabeled datasets,
principled methods to do so for labeled (e.g., classification) datasets are
missing. In this work, we propose a novel framework for dataset transformation,
which we cast as optimization over data-generating joint probability
distributions. We approach this class of problems through Wasserstein gradient
flows in probability space, and derive practical and efficient particle-based
methods for a flexible but well-behaved class of objective functions. Through
various experiments, we show that this framework can be used to impose
constraints on classification datasets, adapt them for transfer learning, or to
re-purpose fixed or black-box models to classify -- with high accuracy --
previously unseen datasets.
Related papers
- Learning from Uncertain Data: From Possible Worlds to Possible Models [13.789554282826835]
We introduce an efficient method for learning linear models from uncertain data, where uncertainty is represented as a set of possible variations in the data.
We compactly represent these dataset variations, enabling the symbolic execution of gradient descent on all possible worlds simultaneously.
Our method provides sound over-approximations of all possible optimal models and viable prediction ranges.
arXiv Detail & Related papers (2024-05-28T19:36:55Z) - Distribution-Aware Data Expansion with Diffusion Models [55.979857976023695]
We propose DistDiff, a training-free data expansion framework based on the distribution-aware diffusion model.
DistDiff consistently enhances accuracy across a diverse range of datasets compared to models trained solely on original data.
arXiv Detail & Related papers (2024-03-11T14:07:53Z) - Joint Distributional Learning via Cramer-Wold Distance [0.7614628596146602]
We introduce the Cramer-Wold distance regularization, which can be computed in a closed-form, to facilitate joint distributional learning for high-dimensional datasets.
We also introduce a two-step learning method to enable flexible prior modeling and improve the alignment between the aggregated posterior and the prior distribution.
arXiv Detail & Related papers (2023-10-25T05:24:23Z) - Geometrically Aligned Transfer Encoder for Inductive Transfer in
Regression Tasks [5.038936775643437]
We propose a novel transfer technique based on differential geometry, namely the Geometrically Aligned Transfer (GATE)
We find a proper diffeomorphism between pairs of tasks to ensure that every arbitrary point maps to a locally flat coordinate in the overlapping region, allowing the transfer of knowledge from the source to the target data.
GATE outperforms conventional methods and exhibits stable behavior in both the latent space and extrapolation regions for various molecular graph datasets.
arXiv Detail & Related papers (2023-10-10T07:11:25Z) - Binary Quantification and Dataset Shift: An Experimental Investigation [54.14283123210872]
Quantification is the supervised learning task that consists of training predictors of the class prevalence values of sets of unlabelled data.
The relationship between quantification and other types of dataset shift remains, by and large, unexplored.
We propose a fine-grained taxonomy of types of dataset shift, by establishing protocols for the generation of datasets affected by these types of shift.
arXiv Detail & Related papers (2023-10-06T20:11:27Z) - Flow Factorized Representation Learning [109.51947536586677]
We introduce a generative model which specifies a distinct set of latent probability paths that define different input transformations.
We show that our model achieves higher likelihoods on standard representation learning benchmarks while simultaneously being closer to approximately equivariant models.
arXiv Detail & Related papers (2023-09-22T20:15:37Z) - A Data-Driven Method for Automated Data Superposition with Applications
in Soft Matter Science [0.0]
We develop a data-driven, non-parametric method for superposing experimental data with arbitrary coordinate transformations.
Our method produces interpretable data-driven models that may inform applications such as materials classification, design, and discovery.
arXiv Detail & Related papers (2022-04-20T14:58:04Z) - Invariance Learning in Deep Neural Networks with Differentiable Laplace
Approximations [76.82124752950148]
We develop a convenient gradient-based method for selecting the data augmentation.
We use a differentiable Kronecker-factored Laplace approximation to the marginal likelihood as our objective.
arXiv Detail & Related papers (2022-02-22T02:51:11Z) - A Lagrangian Duality Approach to Active Learning [119.36233726867992]
We consider the batch active learning problem, where only a subset of the training data is labeled.
We formulate the learning problem using constrained optimization, where each constraint bounds the performance of the model on labeled samples.
We show, via numerical experiments, that our proposed approach performs similarly to or better than state-of-the-art active learning methods.
arXiv Detail & Related papers (2022-02-08T19:18:49Z) - A Trainable Optimal Transport Embedding for Feature Aggregation and its
Relationship to Attention [96.77554122595578]
We introduce a parametrized representation of fixed size, which embeds and then aggregates elements from a given input set according to the optimal transport plan between the set and a trainable reference.
Our approach scales to large datasets and allows end-to-end training of the reference, while also providing a simple unsupervised learning mechanism with small computational cost.
arXiv Detail & Related papers (2020-06-22T08:35:58Z) - Heterogeneous Transfer Learning in Ensemble Clustering [0.0]
We consider a clustering problem in which "similar" labeled data are available.
The method is based on constructing meta-features which describe structural characteristics of data.
An experimental study of the method using Monte Carlo modeling has confirmed its efficiency.
arXiv Detail & Related papers (2020-01-20T16:03:38Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.