Factoring out prior knowledge from low-dimensional embeddings

• URL: http://arxiv.org/abs/2103.01828v1
• Date: Tue, 2 Mar 2021 16:10:36 GMT
• Title: Factoring out prior knowledge from low-dimensional embeddings
• Abstract summary: We propose two methods for factoring out prior knowledge in the form of distance matrices from low-dimensional embeddings. Experiments on both synthetic and real world data show that both methods work well.
• Score: 21.960176662468026
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Low-dimensional embedding techniques such as tSNE and UMAP allow visualizing high-dimensional data and therewith facilitate the discovery of interesting structure. Although they are widely used, they visualize data as is, rather than in light of the background knowledge we have about the data. What we already know, however, strongly determines what is novel and hence interesting. In this paper we propose two methods for factoring out prior knowledge in the form of distance matrices from low-dimensional embeddings. To factor out prior knowledge from tSNE embeddings, we propose JEDI that adapts the tSNE objective in a principled way using Jensen-Shannon divergence. To factor out prior knowledge from any downstream embedding approach, we propose CONFETTI, in which we directly operate on the input distance matrices. Extensive experiments on both synthetic and real world data show that both methods work well, providing embeddings that exhibit meaningful structure that would otherwise remain hidden.

Related papers

• Fuzzy Rule-based Differentiable Representation Learning [16.706014479049493]
  This paper introduces a novel representation learning method grounded in an interpretable fuzzy rule-based model. It is built upon the Takagi-Sugeno-Kang fuzzy system (TSK-FS) to initially map input data to a high-dimensional fuzzy feature space. A novel differentiable optimization method is proposed for the consequence part learning which can preserve the model's interpretability and transparency.
  arXiv  Detail & Related papers  (2025-03-16T14:00:34Z)
• Simple Ingredients for Offline Reinforcement Learning [86.1988266277766]
  offline reinforcement learning algorithms have proven effective on datasets highly connected to the target downstream task. We show that existing methods struggle with diverse data: their performance considerably deteriorates as data collected for related but different tasks is simply added to the offline buffer. We show that scale, more than algorithmic considerations, is the key factor influencing performance.
  arXiv  Detail & Related papers  (2024-03-19T18:57:53Z)
• ExaRanker-Open: Synthetic Explanation for IR using Open-Source LLMs [60.81649785463651]
  We introduce ExaRanker-Open, where we adapt and explore the use of open-source language models to generate explanations. Our findings reveal that incorporating explanations consistently enhances neural rankers, with benefits escalating as the LLM size increases.
  arXiv  Detail & Related papers  (2024-02-09T11:23:14Z)
• Cluster Exploration using Informative Manifold Projections [0.0]
  Dimensionality reduction (DR) is one of the key tools for the visual exploration of high-dimensional data. We propose a novel method to generate informative embeddings which factor out the structure associated with different kinds of prior knowledge.
  arXiv  Detail & Related papers  (2023-09-26T11:35:25Z)
• A Dimensional Structure based Knowledge Distillation Method for Cross-Modal Learning [15.544134849816528]
  We discover the correlation between feature discriminability and dimensional structure (DS) by analyzing and observing features extracted from simple and hard tasks. We propose a novel cross-modal knowledge distillation (CMKD) method for better supervised cross-modal learning (CML) performance. The proposed method enforces output features to be channel-wise independent and intermediate ones to be uniformly distributed, thereby learning semantically irrelevant features from the hard task to boost its accuracy.
  arXiv  Detail & Related papers  (2023-06-28T07:29:26Z)
• A Heat Diffusion Perspective on Geodesic Preserving Dimensionality Reduction [66.21060114843202]
  We propose a more general heat kernel based manifold embedding method that we call heat geodesic embeddings. Results show that our method outperforms existing state of the art in preserving ground truth manifold distances. We also showcase our method on single cell RNA-sequencing datasets with both continuum and cluster structure.
  arXiv  Detail & Related papers  (2023-05-30T13:58:50Z)
• Deep Recursive Embedding for High-Dimensional Data [9.611123249318126]
  We propose to combine deep neural networks (DNN) with mathematics-guided embedding rules for high-dimensional data embedding. We introduce a generic deep embedding network (DEN) framework, which is able to learn a parametric mapping from high-dimensional space to low-dimensional space.
  arXiv  Detail & Related papers  (2021-10-31T23:22:33Z)
• Deep Magnification-Flexible Upsampling over 3D Point Clouds [103.09504572409449]
  We propose a novel end-to-end learning-based framework to generate dense point clouds. We first formulate the problem explicitly, which boils down to determining the weights and high-order approximation errors. Then, we design a lightweight neural network to adaptively learn unified and sorted weights as well as the high-order refinements.
  arXiv  Detail & Related papers  (2020-11-25T14:00:18Z)
• Kernel Two-Dimensional Ridge Regression for Subspace Clustering [45.651770340521786]
  We propose a novel subspace clustering method for 2D data. It directly uses 2D data as inputs such that the learning of representations benefits from inherent structures and relationships of the data.
  arXiv  Detail & Related papers  (2020-11-03T04:52:46Z)
• Two-Dimensional Semi-Nonnegative Matrix Factorization for Clustering [50.43424130281065]
  We propose a new Semi-Nonnegative Matrix Factorization method for 2-dimensional (2D) data, named TS-NMF. It overcomes the drawback of existing methods that seriously damage the spatial information of the data by converting 2D data to vectors in a preprocessing step.
  arXiv  Detail & Related papers  (2020-05-19T05:54:14Z)
• NCVis: Noise Contrastive Approach for Scalable Visualization [79.44177623781043]
  NCVis is a high-performance dimensionality reduction method built on a sound statistical basis of noise contrastive estimation. We show that NCVis outperforms state-of-the-art techniques in terms of speed while preserving the representation quality of other methods.
  arXiv  Detail & Related papers  (2020-01-30T15:43:50Z)
• Multi-Objective Genetic Programming for Manifold Learning: Balancing Quality and Dimensionality [4.4181317696554325]
  State-of-the-art manifold learning algorithms are opaque in how they perform this transformation. We introduce a multi-objective approach that automatically balances the competing objectives of manifold quality and dimensionality. Our proposed approach is competitive with a range of baseline and state-of-the-art manifold learning methods.
  arXiv  Detail & Related papers  (2020-01-05T23:24:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.