Supervised Distributional Reduction via Optimal Transport and Dependence Maximization

• URL: http://arxiv.org/abs/2605.27619v1
• Date: Tue, 26 May 2026 19:38:20 GMT
• Title: Supervised Distributional Reduction via Optimal Transport and Dependence Maximization
• Abstract summary: We propose Supervised Distribution Reduction (SDR), an algorithm for learning target-aware representations.<n>SDR builds on the Fused Gromov-Wasserstein (FGW) objective to align the relational structure of the input distribution with a set of representative points.<n>SDR naturally induces a data-dependent, non-stationary geometry that can be leveraged for settings such as Gaussian Process (GP) modelling.
• Score: 1.0499611180329804
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Learning representations that capture both intrinsic data geometry and target-relevant structure remains a fundamental challenge, particularly in settings where data reduction must balance compression with predictive fidelity. While distributional reduction-encompassing joint clustering and dimensionality reduction-offers a principled way to summarize data, its supervised variants remain relatively under-explored, despite the importance of retaining task-relevant signal for downstream prediction and decision-making. We propose Supervised Distributional Reduction (SDR), an algorithm for learning target-aware representations by combining optimal transport with explicit dependence maximization. SDR builds on the Fused Gromov-Wasserstein (FGW) objective to align the relational structure of the input distribution with a set of representative points, while augmenting it with a direct dependence term that encourages the learned embeddings to capture predictive signal more explicitly. This results in compact representations that reflect both geometric structure and supervision. Beyond representation learning, SDR naturally induces a data-dependent, non-stationary geometry that can be leveraged for settings such as Gaussian Process (GP) modelling. By redefining distances through target-aware distributional alignment, SDR enables the construction of adaptive kernels that respond to local variations in both data geometry and supervision, offering an optimal transport-based perspective on non-stationary kernel design.

Related papers

• ProSR: Semantic-Prototype-Guided Discrete Modeling for Physically Consistent SAR Super-Resolution [28.150128853779268]
  High-resolution Synthetic Aperture Radar (SAR) imagery is critical for precision analysis as automatic target recognition, yet its acquisition is costly.<n>We propose Prototype-Guided Super-Resolution (ProSR) to reformulating SAR ISR as a semantically-guided discrete token prediction task.<n>ProSR achieves superior visual quality while preserving essential scattering characteristics required for practical SAR applications.
  arXiv  Detail & Related papers  (2026-09-02T09:50:06Z)
• Coupled Optimal Transport with Landmark Constraints [64.98367856457412]
  An optimal transport (OT) model seeks a map or plan between distributions by minimizing a prescribed transport cost or distortion.<n>This paper proposes a novel coupled OT framework that leverages a small number of annotated landmarks to guide the recovery of an underlying deformation.<n>The proposed framework provides a principled connection between landmark-based registration and transport-based distribution matching.
  arXiv  Detail & Related papers  (2026-08-20T08:28:13Z)
• RODR: Riemannian Orthogonally Decoupled Regularization for Disentangled Manifold Representation [0.91963810502252]
  We introduce a generic and interpretable framework for disentangled geometric optimization in point cloud processing.<n> Experiments demonstrate that RODR reaches performance comparable to state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-07-27T03:12:14Z)
• The Art of Interrogation: Consistency Amplifies Factuality in Spatial Reasoning [65.67266333751569]
  Current Large Reasoning Models (LRMs) exhibit remarkable general capabilities but significantly underperform in spatial reasoning tasks.<n>We propose a self-supervised reinforcement learning framework that targets the internal reasoning process without requiring ground-truth annotations.<n>We show that this label-free consistency training approaches the accuracy of models trained with ground-truth supervision and achieves similar generalization across diverse tasks and data domains.
  arXiv  Detail & Related papers  (2026-06-10T10:50:06Z)
• Geometry-Aware Dataset Condensation for Diffusion Model Training [103.45641113998839]
  We propose to reformulate real subset selection as a geometry-aware distribution alignment problem.<n>By incorporating one-sided partial optimal transport, our method selectively aligns a compact subset with the full data distribution.<n>Experiments across diffusion variants, subset sizes, image resolutions, and training rounds show that our method achieves superior fidelity and distributional coverage.
  arXiv  Detail & Related papers  (2026-06-04T08:53:58Z)
• Beyond Point-Wise Matching: Structural Representation Alignment for Accelerating Diffusion Transformers [93.3976834364707]
  We propose sREPA, a structural REPresentation Alignment framework to enforce consistency in the relational geometry of feature maps.<n>By encouraging the model to internalize holistic spatial layouts and structural correlations from pre-trained features, sREPA achieves faster and more stable convergence.
  arXiv  Detail & Related papers  (2026-05-16T12:01:04Z)
• SEED: Targeted Data Selection by Weighted Independent Set [76.68391670109433]
  We develop a robust and scalable data selection pipeline dubbed SEED.<n>SEED consistently outperforms state-of-the-art methods on instruction tuning, visual instruction tuning, and semantic segmentation.
  arXiv  Detail & Related papers  (2026-05-15T07:26:54Z)
• Geometric Characterisation and Structured Trajectory Surrogates for Clinical Dataset Condensation [1.6261700844888172]
  Trajectory matching (TM) is a widely used condensation approach that supervises synthetic data using changes in model parameters observed during training on real data.<n>We show that a fixed synthetic dataset can only reproduce a limited span of such training-induced parameter changes.<n>Motivated by this mismatch, we propose Bezier Trajectory Matching (BTM), which replaces SGD trajectories with quadratic Bezier trajectory surrogates.
  arXiv  Detail & Related papers  (2026-04-23T12:53:46Z)
• Curvature-Aware PCA with Geodesic Tangent Space Aggregation for Semi-Supervised Learning [52.452902154360565]
  GTSA-PCA is a geometric extension of Principal Component Analysis.<n>It integrates curvature awareness and geodesic consistency within a unified spectral framework.<n>Our results position GTSA-PCA as a principled bridge between statistical and geometric approaches to dimensionality reduction.
  arXiv  Detail & Related papers  (2026-04-20T20:36:36Z)
• Invariance on Manifolds: Understanding Robust Visual Representations for Place Recognition [19.200074425090595]
  We propose a Second-Order Geometric Statistics framework that inherently captures geometric stability without training.<n>Our approach introduces a training-free framework built upon fixed, pre-trained backbones, achieving strong zero-shot generalization without parameter updates.
  arXiv  Detail & Related papers  (2026-01-31T18:12:29Z)
• Designing Ambiguity Sets for Distributionally Robust Optimization Using Structural Causal Optimal Transport [21.387312729118364]
  We propose incorporating structural equations, which include causal graph information, to enhance ambiguity sets.<n>We show how our method overcomes the curse of dimensionality in optimal transport problems, achieving faster shrinkage with-free order.
  arXiv  Detail & Related papers  (2025-10-01T07:26:47Z)
• Representation-Aware Distributionally Robust Optimization: A Knowledge Transfer Framework [6.529107536201152]
  We propose a novel framework for Wasserstein distributionally robust learning that accounts for predictive representations when guarding against distributional shifts.<n>We show that READ embeds a multidimensional alignment parameter into the transport cost, allowing the model to differentially discourage perturbations along directions associated with informative representations.<n>We conclude by demonstrating the effectiveness of our framework through extensive simulations and a real-world study.
  arXiv  Detail & Related papers  (2025-09-11T11:42:17Z)
• Compressed Regression over Adaptive Networks [58.79251288443156]
  We derive the performance achievable by a network of distributed agents that solve, adaptively and in the presence of communication constraints, a regression problem. We devise an optimized allocation strategy where the parameters necessary for the optimization can be learned online by the agents.
  arXiv  Detail & Related papers  (2023-04-07T13:41:08Z)
• Which Invariance Should We Transfer? A Causal Minimax Learning Approach [18.71316951734806]
  We present a comprehensive minimax analysis from a causal perspective. We propose an efficient algorithm to search for the subset with minimal worst-case risk. The effectiveness and efficiency of our methods are demonstrated on synthetic data and the diagnosis of Alzheimer's disease.
  arXiv  Detail & Related papers  (2021-07-05T09:07:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.