Decorrelation using Optimal Transport

• URL: http://arxiv.org/abs/2307.05187v2
• Date: Fri, 14 Jul 2023 07:24:54 GMT
• Title: Decorrelation using Optimal Transport
• Authors: Malte Algren, John Andrew Raine and Tobias Golling
• Abstract summary: We introduce a novel decorrelation method that is able to decorrelate a continuous feature space against protected attributes with optimal transport. We demonstrate how well it performs in the context of jet classification in high energy physics. When moving to multiclass outputs the optimal transport approach performs significantly better than the state-of-the-art, suggesting substantial gains at decorrelating multidimensional feature spaces.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Being able to decorrelate a feature space from protected attributes is an area of active research and study in ethics, fairness, and also natural sciences. We introduce a novel decorrelation method using Convex Neural Optimal Transport Solvers (Cnots) that is able to decorrelate a continuous feature space against protected attributes with optimal transport. We demonstrate how well it performs in the context of jet classification in high energy physics, where classifier scores are desired to be decorrelated from the mass of a jet. The decorrelation achieved in binary classification approaches the levels achieved by the state-of-the-art using conditional normalising flows. When moving to multiclass outputs the optimal transport approach performs significantly better than the state-of-the-art, suggesting substantial gains at decorrelating multidimensional feature spaces.

Related papers

• Simultaneous Learning of Optimal Transports for Training All-to-All Flow-Based Condition Transfer Model [19.71452214879951]
  We introduce a novel cost function that enables simultaneous learning of optimal transports for all pairs of conditional distributions. Our method is supported by a theoretical guarantee that, in the limit, it converges to pairwise optimal transports among infinite pairs of conditional distributions. The learned transport maps are subsequently used to couple data points in conditional flow matching.
  arXiv  Detail & Related papers  (2025-04-04T05:32:54Z)
• Partial Distribution Alignment via Adaptive Optimal Transport [11.167177175327359]
  We propose adaptive optimal transport which is distinctive from the classical optimal transport in its ability of adaptive-mass preserving. We instantiate the adaptive optimal transport in machine learning application to align source and target distributions partially and adaptively.
  arXiv  Detail & Related papers  (2025-03-07T02:13:04Z)
• I3S: Importance Sampling Subspace Selection for Low-Rank Optimization in LLM Pretraining [50.89661053183944]
  Low-rank optimization has emerged as a promising approach to enabling memory-efficient training of large language models (LLMs) Existing low-rank optimization methods typically project gradients onto a low-rank subspace, reducing the memory cost of storing states. We propose importance sampling subspace selection (I3S) for low-rank optimization, which theoretically offers a comparable convergence rate to the dominant subspace approach.
  arXiv  Detail & Related papers  (2025-02-09T06:30:19Z)
• Preventing Local Pitfalls in Vector Quantization via Optimal Transport [77.15924044466976]
  We introduce OptVQ, a novel vector quantization method that employs the Sinkhorn algorithm to optimize the optimal transport problem. Our experiments on image reconstruction tasks demonstrate that OptVQ achieves 100% codebook utilization and surpasses current state-of-the-art VQNs in reconstruction quality.
  arXiv  Detail & Related papers  (2024-12-19T18:58:14Z)
• Optimal Transport Maps are Good Voice Converters [58.42556113055807]
  We present a variety of optimal transport algorithms for different data representations, such as mel-spectrograms and latent representation of self-supervised speech models. For the mel-spectogram data representation, we achieve strong results in terms of Frechet Audio Distance (FAD) We achived state-of-the-art results and outperformed existing methods even with limited reference speaker data.
  arXiv  Detail & Related papers  (2024-10-17T22:48:53Z)
• Unbalanced Optimal Transport: A Unified Framework for Object Detection [97.74382560746987]
  We show how Unbalanced Optimal Transport unifies different approaches to object detection. We show that training an object detection model with Unbalanced Optimal Transport is able to reach the state-of-the-art. The approach is well suited for GPU implementation, which proves to be an advantage for large-scale models.
  arXiv  Detail & Related papers  (2023-07-05T16:21:52Z)
• Estimator: An Effective and Scalable Framework for Transportation Mode Classification over Trajectories [22.704053034976784]
  We propose an effective and scalable framework for transportation mode classification over GPS trajectories, abbreviated Estimator. Estimator partitions the entire traffic space into disjointed spatial regions according to traffic conditions, which enhances the scalability significantly. We show that Estimator achieves superior model effectiveness (i.e., 99% Accuracy and 0.98 F1-score), which outperforms state-of-the-arts learning-based methods.
  arXiv  Detail & Related papers  (2022-12-11T13:32:49Z)
• Bi-Level Optimization Augmented with Conditional Variational Autoencoder for Autonomous Driving in Dense Traffic [0.9281671380673306]
  This paper presents a parameterized bi-level optimization that jointly computes the optimal behavioural decisions and the resulting trajectory. Our approach runs in real-time using a custom GPU-accelerated batch, and a Variational Autoencoder learnt warm-start strategy. Our approach outperforms state-of-the-art model predictive control and RL approaches in terms of collision rate while being competitive in driving efficiency.
  arXiv  Detail & Related papers  (2022-12-05T12:56:42Z)
• InfoOT: Information Maximizing Optimal Transport [58.72713603244467]
  InfoOT is an information-theoretic extension of optimal transport. It maximizes the mutual information between domains while minimizing geometric distances. This formulation yields a new projection method that is robust to outliers and generalizes to unseen samples.
  arXiv  Detail & Related papers  (2022-10-06T18:55:41Z)
• Manifold Interpolating Optimal-Transport Flows for Trajectory Inference [64.94020639760026]
  We present a method called Manifold Interpolating Optimal-Transport Flow (MIOFlow) MIOFlow learns, continuous population dynamics from static snapshot samples taken at sporadic timepoints. We evaluate our method on simulated data with bifurcations and merges, as well as scRNA-seq data from embryoid body differentiation, and acute myeloid leukemia treatment.
  arXiv  Detail & Related papers  (2022-06-29T22:19:03Z)
• Constrained Mass Optimal Transport [0.0]
  This paper introduces the problem of constrained optimal transport. A family of algorithms is introduced to solve a class of constrained saddle point problems. Convergence proofs and numerical results are presented.
  arXiv  Detail & Related papers  (2022-06-05T06:47:25Z)
• On the Existence of Optimal Transport Gradient for Learning Generative Models [8.602553195689513]
  Training of Wasserstein Generative Adversarial Networks (WGAN) relies on the calculation of the gradient of the optimal transport cost. We first demonstrate that such gradient may not be defined, which can result in numerical instabilities during gradient-based optimization. By exploiting the discrete nature of empirical data, we formulate the gradient in a semi-discrete setting and propose an algorithm for the optimization of the generative model parameters.
  arXiv  Detail & Related papers  (2021-02-10T16:28:20Z)
• A Trainable Optimal Transport Embedding for Feature Aggregation and its Relationship to Attention [96.77554122595578]
  We introduce a parametrized representation of fixed size, which embeds and then aggregates elements from a given input set according to the optimal transport plan between the set and a trainable reference. Our approach scales to large datasets and allows end-to-end training of the reference, while also providing a simple unsupervised learning mechanism with small computational cost.
  arXiv  Detail & Related papers  (2020-06-22T08:35:58Z)
• Large-Scale Optimal Transport via Adversarial Training with Cycle-Consistency [30.305690062622283]
  We propose an end-to-end approach for large-scale optimal transport, which directly solves the transport map and is compatible with general cost function. We demonstrate the effectiveness of the proposed method against existing methods with large-scale real-world applications.
  arXiv  Detail & Related papers  (2020-03-14T14:06:46Z)
• TrajectoryNet: A Dynamic Optimal Transport Network for Modeling Cellular Dynamics [74.43710101147849]
  We present TrajectoryNet, which controls the continuous paths taken between distributions to produce dynamic optimal transport. We show how this is particularly applicable for studying cellular dynamics in data from single-cell RNA sequencing (scRNA-seq) technologies.
  arXiv  Detail & Related papers  (2020-02-09T21:00:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.