The Shape of Attraction in UMAP: Exploring the Embedding Forces in Dimensionality Reduction

• URL: http://arxiv.org/abs/2503.09101v2
• Date: Tue, 18 Mar 2025 15:48:38 GMT
• Title: The Shape of Attraction in UMAP: Exploring the Embedding Forces in Dimensionality Reduction
• Authors: Mohammad Tariqul Islam, Jason W. Fleischer,
• Abstract summary: We analyze the forces to reveal their effects on cluster formations and visualization.<n>Our analysis makes UMAP and similar embedding methods more interpretable, more robust, and more accurate.
• Score: 1.206248959194646
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Uniform manifold approximation and projection (UMAP) is among the most popular neighbor embedding methods. The method relies on attractive and repulsive forces among high-dimensional data points to obtain a low-dimensional embedding. In this paper, we analyze the forces to reveal their effects on cluster formations and visualization. Repulsion emphasizes differences, controlling cluster boundaries and inter-cluster distance. Attraction is more subtle, as attractive tension between points can manifest simultaneously as attraction and repulsion in the lower-dimensional mapping. This explains the need for learning rate annealing and motivates the different treatments between attractive and repulsive terms. Moreover, by modifying attraction, we improve the consistency of cluster formation under random initialization. Overall, our analysis makes UMAP and similar embedding methods more interpretable, more robust, and more accurate.

Related papers

• Attraction-Repulsion Swarming: A Generalized Framework of t-SNE via Force Normalization and Tunable Interactions [2.3020018305241337]
  ARS is a framework that is based on viewing the t-distributed data neighbor embedding (t-SNE) visualization technique as a swarm of interacting agents driven by attraction and repulsion forces. ARS also includes the ability to separately tune the attraction and repulsion kernels, which gives the user control over the tightness within clusters and the spacing between them in the visualization.
  arXiv  Detail & Related papers  (2024-11-15T22:42:11Z)
• Enhancing Counterfactual Explanation Search with Diffusion Distance and Directional Coherence [0.0]
  A pressing issue in the adoption of AI models is the increasing demand for more human-centric explanations of their predictions. We propose and test the incorporation of two novel biases to enhance the search for effective counterfactual explanations.
  arXiv  Detail & Related papers  (2024-04-19T11:47:17Z)
• Deep Clustering with Diffused Sampling and Hardness-aware Self-distillation [4.550555443103878]
  This paper proposes a novel end-to-end deep clustering method with diffused sampling and hardness-aware self-distillation (HaDis) Results on five challenging image datasets demonstrate the superior clustering performance of our HaDis method over the state-of-the-art.
  arXiv  Detail & Related papers  (2024-01-25T09:33:49Z)
• Hierarchical Compositional Representations for Few-shot Action Recognition [51.288829293306335]
  We propose a novel hierarchical compositional representations (HCR) learning approach for few-shot action recognition. We divide a complicated action into several sub-actions by carefully designed hierarchical clustering. We also adopt the Earth Mover's Distance in the transportation problem to measure the similarity between video samples in terms of sub-action representations.
  arXiv  Detail & Related papers  (2022-08-19T16:16:59Z)
• An Embedding-Dynamic Approach to Self-supervised Learning [8.714677279673738]
  We treat the embeddings of images as point particles and consider model optimization as a dynamic process on this system of particles. Our dynamic model combines an attractive force for similar images, a locally dispersive force to avoid local collapse, and a global dispersive force to achieve a globally-homogeneous distribution of particles.
  arXiv  Detail & Related papers  (2022-07-07T19:56:20Z)
• PANet: Perspective-Aware Network with Dynamic Receptive Fields and Self-Distilling Supervision for Crowd Counting [63.84828478688975]
  We propose a novel perspective-aware approach called PANet to address the perspective problem. Based on the observation that the size of the objects varies greatly in one image due to the perspective effect, we propose the dynamic receptive fields (DRF) framework. The framework is able to adjust the receptive field by the dilated convolution parameters according to the input image, which helps the model to extract more discriminative features for each local region.
  arXiv  Detail & Related papers  (2021-10-31T04:43:05Z)
• Hard-label Manifolds: Unexpected Advantages of Query Efficiency for Finding On-manifold Adversarial Examples [67.23103682776049]
  Recent zeroth order hard-label attacks on image classification models have shown comparable performance to their first-order, gradient-level alternatives. It was recently shown in the gradient-level setting that regular adversarial examples leave the data manifold, while their on-manifold counterparts are in fact generalization errors. We propose an information-theoretic argument based on a noisy manifold distance oracle, which leaks manifold information through the adversary's gradient estimate.
  arXiv  Detail & Related papers  (2021-03-04T20:53:06Z)
• Deep Magnification-Flexible Upsampling over 3D Point Clouds [103.09504572409449]
  We propose a novel end-to-end learning-based framework to generate dense point clouds. We first formulate the problem explicitly, which boils down to determining the weights and high-order approximation errors. Then, we design a lightweight neural network to adaptively learn unified and sorted weights as well as the high-order refinements.
  arXiv  Detail & Related papers  (2020-11-25T14:00:18Z)
• Fast Gravitational Approach for Rigid Point Set Registration with Ordinary Differential Equations [79.71184760864507]
  This article introduces a new physics-based method for rigid point set alignment called Fast Gravitational Approach (FGA) In FGA, the source and target point sets are interpreted as rigid particle swarms with masses interacting in a globally multiply-linked manner while moving in a simulated gravitational force field. We show that the new method class has characteristics not found in previous alignment methods.
  arXiv  Detail & Related papers  (2020-09-28T15:05:39Z)
• Attraction-Repulsion Spectrum in Neighbor Embeddings [6.129463540742259]
  Neighbor embedding algorithms combine an attractive force between neighboring pairs of points with a repulsive force between all points. Here we empirically show that changing the balance between the attractive and the repulsive forces in t-SNE using the exaggeration parameter yields a spectrum of embeddings.
  arXiv  Detail & Related papers  (2020-07-17T11:10:04Z)
• Disentangling Adaptive Gradient Methods from Learning Rates [65.0397050979662]
  We take a deeper look at how adaptive gradient methods interact with the learning rate schedule. We introduce a "grafting" experiment which decouples an update's magnitude from its direction. We present some empirical and theoretical retrospectives on the generalization of adaptive gradient methods.
  arXiv  Detail & Related papers  (2020-02-26T21:42:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.