Lightspeed Geometric Dataset Distance via Sliced Optimal Transport

• URL: http://arxiv.org/abs/2501.18901v1
• Date: Fri, 31 Jan 2025 05:42:58 GMT
• Title: Lightspeed Geometric Dataset Distance via Sliced Optimal Transport
• Authors: Khai Nguyen, Hai Nguyen, Tuan Pham, Nhat Ho,
• Abstract summary: We introduce sliced optimal transport dataset distance (s-OTDD), a model-agnostic, embedding-agnostic approach for dataset comparison. We derive a data point projection that transforms datasets into one-dimensional distributions.
• Score: 35.22009725098762
• License:
• Abstract: We introduce sliced optimal transport dataset distance (s-OTDD), a model-agnostic, embedding-agnostic approach for dataset comparison that requires no training, is robust to variations in the number of classes, and can handle disjoint label sets. The core innovation is Moment Transform Projection (MTP), which maps a label, represented as a distribution over features, to a real number. Using MTP, we derive a data point projection that transforms datasets into one-dimensional distributions. The s-OTDD is defined as the expected Wasserstein distance between the projected distributions, with respect to random projection parameters. Leveraging the closed form solution of one-dimensional optimal transport, s-OTDD achieves (near-)linear computational complexity in the number of data points and feature dimensions and is independent of the number of classes. With its geometrically meaningful projection, s-OTDD strongly correlates with the optimal transport dataset distance while being more efficient than existing dataset discrepancy measures. Moreover, it correlates well with the performance gap in transfer learning and classification accuracy in data augmentation.

Related papers

• Swift Cross-Dataset Pruning: Enhancing Fine-Tuning Efficiency in Natural Language Understanding [2.379669478864599]
  Current cross-dataset pruning techniques for fine-tuning often rely on computationally expensive sample ranking processes. We propose Swift Cross-Dataset Pruning (SCDP), which uses TF-IDF embeddings with geometric median to rapidly evaluate sample importance. Experimental results on six diverse datasets demonstrate the effectiveness of our method, spanning various tasks and scales.
  arXiv  Detail & Related papers  (2025-01-05T03:52:04Z)
• TAROT: Targeted Data Selection via Optimal Transport [64.56083922130269]
  TAROT is a targeted data selection framework grounded in optimal transport theory. Previous targeted data selection methods rely on influence-based greedys to enhance domain-specific performance. We evaluate TAROT across multiple tasks, including semantic segmentation, motion prediction, and instruction tuning.
  arXiv  Detail & Related papers  (2024-11-30T10:19:51Z)
• Minimally Supervised Learning using Topological Projections in Self-Organizing Maps [55.31182147885694]
  We introduce a semi-supervised learning approach based on topological projections in self-organizing maps (SOMs) Our proposed method first trains SOMs on unlabeled data and then a minimal number of available labeled data points are assigned to key best matching units (BMU) Our results indicate that the proposed minimally supervised model significantly outperforms traditional regression techniques.
  arXiv  Detail & Related papers  (2024-01-12T22:51:48Z)
• Improved Distribution Matching for Dataset Condensation [91.55972945798531]
  We propose a novel dataset condensation method based on distribution matching. Our simple yet effective method outperforms most previous optimization-oriented methods with much fewer computational resources.
  arXiv  Detail & Related papers  (2023-07-19T04:07:33Z)
• InfoOT: Information Maximizing Optimal Transport [58.72713603244467]
  InfoOT is an information-theoretic extension of optimal transport. It maximizes the mutual information between domains while minimizing geometric distances. This formulation yields a new projection method that is robust to outliers and generalizes to unseen samples.
  arXiv  Detail & Related papers  (2022-10-06T18:55:41Z)
• Uniform-in-Phase-Space Data Selection with Iterative Normalizing Flows [0.0]
  A strategy is proposed to select data points such that they uniformly span the phase-space of the data. An iterative method is used to accurately estimate the probability of the rare data points when only a small subset of the dataset is used to construct the probability map. The proposed framework is demonstrated as a viable pathway to enable data-efficient machine learning when abundant data is available.
  arXiv  Detail & Related papers  (2021-12-28T20:06:28Z)
• Near-optimal estimation of smooth transport maps with kernel sums-of-squares [81.02564078640275]
  Under smoothness conditions, the squared Wasserstein distance between two distributions could be efficiently computed with appealing statistical error upper bounds. The object of interest for applications such as generative modeling is the underlying optimal transport map. We propose the first tractable algorithm for which the statistical $L2$ error on the maps nearly matches the existing minimax lower-bounds for smooth map estimation.
  arXiv  Detail & Related papers  (2021-12-03T13:45:36Z)
• An Information-Geometric Distance on the Space of Tasks [31.359578768463752]
  This paper prescribes a distance between learning tasks modeled as joint distributions on data and labels. We develop an algorithm to compute the distance which iteratively transports the marginal on the data of the source task to that of the target task. We perform thorough empirical validation and analysis across diverse image classification datasets to show that the coupled transfer distance correlates strongly with the difficulty of fine-tuning.
  arXiv  Detail & Related papers  (2020-11-01T19:48:39Z)
• Geometric Dataset Distances via Optimal Transport [15.153110906331733]
  We propose an alternative notion of distance between datasets that (i) is model-agnostic, (ii) does not involve training, (iii) can compare datasets even if their label sets are completely disjoint and (iv) has solid theoretical footing. This distance relies on optimal transport, which provides it with rich geometry awareness, interpretable correspondences and well-understood properties. Our results show that this novel distance provides meaningful comparison of datasets, and correlates well with transfer learning hardness across various experimental settings and datasets.
  arXiv  Detail & Related papers  (2020-02-07T17:51:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.