Supervised learning of sheared distributions using linearized optimal transport

• URL: http://arxiv.org/abs/2201.10590v1
• Date: Tue, 25 Jan 2022 19:19:59 GMT
• Title: Supervised learning of sheared distributions using linearized optimal transport
• Authors: Varun Khurana, Harish Kannan, Alexander Cloninger, Caroline Moosm\"uller
• Abstract summary: In this paper we study supervised learning tasks on the space of probability measures. We approach this problem by embedding the space of probability measures into $L2$ spaces using the optimal transport framework. Regular machine learning techniques are used to achieve linear separability.
• Score: 64.53761005509386
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper we study supervised learning tasks on the space of probability measures. We approach this problem by embedding the space of probability measures into $L^2$ spaces using the optimal transport framework. In the embedding spaces, regular machine learning techniques are used to achieve linear separability. This idea has proved successful in applications and when the classes to be separated are generated by shifts and scalings of a fixed measure. This paper extends the class of elementary transformations suitable for the framework to families of shearings, describing conditions under which two classes of sheared distributions can be linearly separated. We furthermore give necessary bounds on the transformations to achieve a pre-specified separation level, and show how multiple embeddings can be used to allow for larger families of transformations. We demonstrate our results on image classification tasks.

Related papers

• Transformation-Invariant Learning and Theoretical Guarantees for OOD Generalization [34.036655200677664]
  This paper focuses on a distribution shift setting where train and test distributions can be related by classes of (data) transformation maps. We establish learning rules and algorithmic reductions to Empirical Risk Minimization (ERM) We highlight that the learning rules we derive offer a game-theoretic viewpoint on distribution shift.
  arXiv  Detail & Related papers  (2024-10-30T20:59:57Z)
• SimO Loss: Anchor-Free Contrastive Loss for Fine-Grained Supervised Contrastive Learning [0.0]
  We introduce a novel anchor-free contrastive learning (L) method leveraging our proposed Similarity-Orthogonality (SimO) loss. Our approach minimizes a semi-metric discriminative loss function that simultaneously optimize two key objectives. We provide visualizations that demonstrate the impact of SimO loss on the embedding space.
  arXiv  Detail & Related papers  (2024-10-07T17:41:10Z)
• Linear optimal transport subspaces for point set classification [12.718843888673227]
  We propose a framework for classifying point sets experiencing certain types of spatial deformations. Our approach employs the Linear Optimal Transport (LOT) transform to obtain a linear embedding of set-structured data. It achieves competitive accuracies compared to state-of-the-art methods across various point set classification tasks.
  arXiv  Detail & Related papers  (2024-03-15T04:39:27Z)
• Learning representations that are closed-form Monge mapping optimal with application to domain adaptation [24.258758784011572]
  Optimal transport (OT) is a powerful tool used to compare and align probability measures following the least effort principle. Despite its widespread use in machine learning (ML), OT problem still bears its computational burden. We propose to tackle these challenges using representation learning.
  arXiv  Detail & Related papers  (2023-05-12T14:14:39Z)
• Unsupervised Manifold Linearizing and Clustering [19.879641608165887]
  We propose to optimize the Maximal Coding Reduction metric with respect to both the data representation and a novel doubly cluster membership. Experiments on CIFAR-10, -20, -100, and TinyImageNet-200 datasets show that the proposed method is much more accurate and scalable than state-of-the-art deep clustering methods.
  arXiv  Detail & Related papers  (2023-01-04T20:08:23Z)
• Semi-supervised Domain Adaptive Structure Learning [72.01544419893628]
  Semi-supervised domain adaptation (SSDA) is a challenging problem requiring methods to overcome both 1) overfitting towards poorly annotated data and 2) distribution shift across domains. We introduce an adaptive structure learning method to regularize the cooperation of SSL and DA.
  arXiv  Detail & Related papers  (2021-12-12T06:11:16Z)
• Reinforcement Learning in Factored Action Spaces using Tensor Decompositions [92.05556163518999]
  We propose a novel solution for Reinforcement Learning (RL) in large, factored action spaces using tensor decompositions. We use cooperative multi-agent reinforcement learning scenario as the exemplary setting.
  arXiv  Detail & Related papers  (2021-10-27T15:49:52Z)
• The Advantage of Conditional Meta-Learning for Biased Regularization and Fine-Tuning [50.21341246243422]
  Biased regularization and fine-tuning are two recent meta-learning approaches. We propose conditional meta-learning, inferring a conditioning function mapping task's side information into a meta- parameter vector. We then propose a convex meta-algorithm providing a comparable advantage also in practice.
  arXiv  Detail & Related papers  (2020-08-25T07:32:16Z)
• A Trainable Optimal Transport Embedding for Feature Aggregation and its Relationship to Attention [96.77554122595578]
  We introduce a parametrized representation of fixed size, which embeds and then aggregates elements from a given input set according to the optimal transport plan between the set and a trainable reference. Our approach scales to large datasets and allows end-to-end training of the reference, while also providing a simple unsupervised learning mechanism with small computational cost.
  arXiv  Detail & Related papers  (2020-06-22T08:35:58Z)
• Rethinking preventing class-collapsing in metric learning with margin-based losses [81.22825616879936]
  Metric learning seeks embeddings where visually similar instances are close and dissimilar instances are apart. margin-based losses tend to project all samples of a class onto a single point in the embedding space. We propose a simple modification to the embedding losses such that each sample selects its nearest same-class counterpart in a batch.
  arXiv  Detail & Related papers  (2020-06-09T09:59:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.