Manifold Learning via Foliations and Knowledge Transfer

• URL: http://arxiv.org/abs/2409.07412v1
• Date: Wed, 11 Sep 2024 16:53:53 GMT
• Title: Manifold Learning via Foliations and Knowledge Transfer
• Authors: E. Tron, E. Fioresi,
• Abstract summary: We provide a natural geometric structure on the space of data employing a deep ReLU neural network trained as a classifier. We show that the singular points of such foliation are contained in a measure zero set, and that a local regular foliation exists almost everywhere. Experiments show that the data is correlated with leaves of such foliation.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Understanding how real data is distributed in high dimensional spaces is the key to many tasks in machine learning. We want to provide a natural geometric structure on the space of data employing a deep ReLU neural network trained as a classifier. Through the data information matrix (DIM), a variation of the Fisher information matrix, the model will discern a singular foliation structure on the space of data. We show that the singular points of such foliation are contained in a measure zero set, and that a local regular foliation exists almost everywhere. Experiments show that the data is correlated with leaves of such foliation. Moreover we show the potential of our approach for knowledge transfer by analyzing the spectrum of the DIM to measure distances between datasets.

Related papers

• Neural FIM for learning Fisher Information Metrics from point cloud data [71.07939200676199]
  We propose neural FIM, a method for computing the Fisher information metric (FIM) from point cloud data. We demonstrate its utility in selecting parameters for the PHATE visualization method as well as its ability to obtain information pertaining to local volume illuminating branching points and cluster centers embeddings of a toy dataset and two single-cell datasets of IPSC reprogramming and PBMCs (immune cells)
  arXiv  Detail & Related papers  (2023-06-01T17:36:13Z)
• Study of Manifold Geometry using Multiscale Non-Negative Kernel Graphs [32.40622753355266]
  We propose a framework to study the geometric structure of the data. We make use of our recently introduced non-negative kernel (NNK) regression graphs to estimate the point density, intrinsic dimension, and the linearity of the data manifold (curvature)
  arXiv  Detail & Related papers  (2022-10-31T17:01:17Z)
• The Manifold Hypothesis for Gradient-Based Explanations [55.01671263121624]
  gradient-based explanation algorithms provide perceptually-aligned explanations. We show that the more a feature attribution is aligned with the tangent space of the data, the more perceptually-aligned it tends to be. We suggest that explanation algorithms should actively strive to align their explanations with the data manifold.
  arXiv  Detail & Related papers  (2022-06-15T08:49:24Z)
• CHALLENGER: Training with Attribution Maps [63.736435657236505]
  We show that utilizing attribution maps for training neural networks can improve regularization of models and thus increase performance. In particular, we show that our generic domain-independent approach yields state-of-the-art results in vision, natural language processing and on time series tasks.
  arXiv  Detail & Related papers  (2022-05-30T13:34:46Z)
• SubOmiEmbed: Self-supervised Representation Learning of Multi-omics Data for Cancer Type Classification [4.992154875028543]
  Integration and analysis of multi-omics data give us a broad view of tumours, which can improve clinical decision making. SubOmiEmbed produces comparable results to the baseline OmiEmbed with a much smaller network and by using just a subset of the data. This work can be improved to integrate mutation-based genomic data as well.
  arXiv  Detail & Related papers  (2022-02-03T16:39:09Z)
• Learning Signal-Agnostic Manifolds of Neural Fields [50.066449953522685]
  We leverage neural fields to capture the underlying structure in image, shape, audio and cross-modal audiovisual domains. We show that by walking across the underlying manifold of GEM, we may generate new samples in our signal domains.
  arXiv  Detail & Related papers  (2021-11-11T18:57:40Z)
• Reasoning-Modulated Representations [85.08205744191078]
  We study a common setting where our task is not purely opaque. Our approach paves the way for a new class of data-efficient representation learning.
  arXiv  Detail & Related papers  (2021-07-19T13:57:13Z)
• MESS: Manifold Embedding Motivated Super Sampling [0.0]
  We propose a framework to generate virtual data points that faithful to an approximate embedding function underlying the manifold observable in the data. For increasing intrinsic dimensionality of a data set the required data density introduces the need for very large data sets.
  arXiv  Detail & Related papers  (2021-07-14T09:07:54Z)
• Model-centric Data Manifold: the Data Through the Eyes of the Model [0.0]
  Deep ReLU neural network classifiers can see a low-dimensional manifold structure on data. We show that the dataset on which the model is trained lies on a leaf, the data leaf, whose dimension is bounded by the number of classification labels.
  arXiv  Detail & Related papers  (2021-04-26T16:03:09Z)
• Graph Embedding with Data Uncertainty [113.39838145450007]
  spectral-based subspace learning is a common data preprocessing step in many machine learning pipelines. Most subspace learning methods do not take into consideration possible measurement inaccuracies or artifacts that can lead to data with high uncertainty.
  arXiv  Detail & Related papers  (2020-09-01T15:08:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.