Genetic Programming for Manifold Learning: Preserving Local Topology
- URL: http://arxiv.org/abs/2108.09914v1
- Date: Mon, 23 Aug 2021 03:48:48 GMT
- Title: Genetic Programming for Manifold Learning: Preserving Local Topology
- Authors: Andrew Lensen, Bing Xue, Mengjie Zhang
- Abstract summary: We propose a new approach to using genetic programming for manifold learning, which preserves local topology.
This is expected to significantly improve performance on tasks where local neighbourhood structure (topology) is paramount.
- Score: 5.226724669049025
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Manifold learning methods are an invaluable tool in today's world of
increasingly huge datasets. Manifold learning algorithms can discover a much
lower-dimensional representation (embedding) of a high-dimensional dataset
through non-linear transformations that preserve the most important structure
of the original data. State-of-the-art manifold learning methods directly
optimise an embedding without mapping between the original space and the
discovered embedded space. This makes interpretability - a key requirement in
exploratory data analysis - nearly impossible. Recently, genetic programming
has emerged as a very promising approach to manifold learning by evolving
functional mappings from the original space to an embedding. However, genetic
programming-based manifold learning has struggled to match the performance of
other approaches. In this work, we propose a new approach to using genetic
programming for manifold learning, which preserves local topology. This is
expected to significantly improve performance on tasks where local
neighbourhood structure (topology) is paramount. We compare our proposed
approach with various baseline manifold learning methods and find that it often
outperforms other methods, including a clear improvement over previous genetic
programming approaches. These results are particularly promising, given the
potential interpretability and reusability of the evolved mappings.
Related papers
- Genetic Programming for Explainable Manifold Learning [2.370068482059863]
We introduce Genetic Programming for Explainable Manifold Learning (GP-EMaL), a novel approach that directly penalises tree complexity.
Our new method is able to maintain high manifold quality while significantly enhancing explainability and also allows customisation of complexity measures.
arXiv Detail & Related papers (2024-03-21T05:17:22Z) - Supervised Manifold Learning via Random Forest Geometry-Preserving
Proximities [0.0]
We show the weaknesses of class-conditional manifold learning methods quantitatively and visually.
We propose an alternate choice of kernel for supervised dimensionality reduction using a data-geometry-preserving variant of random forest proximities.
arXiv Detail & Related papers (2023-07-03T14:55:11Z) - Object Representations as Fixed Points: Training Iterative Refinement
Algorithms with Implicit Differentiation [88.14365009076907]
Iterative refinement is a useful paradigm for representation learning.
We develop an implicit differentiation approach that improves the stability and tractability of training.
arXiv Detail & Related papers (2022-07-02T10:00:35Z) - Stabilizing Q-learning with Linear Architectures for Provably Efficient
Learning [53.17258888552998]
This work proposes an exploration variant of the basic $Q$-learning protocol with linear function approximation.
We show that the performance of the algorithm degrades very gracefully under a novel and more permissive notion of approximation error.
arXiv Detail & Related papers (2022-06-01T23:26:51Z) - Inducing Gaussian Process Networks [80.40892394020797]
We propose inducing Gaussian process networks (IGN), a simple framework for simultaneously learning the feature space as well as the inducing points.
The inducing points, in particular, are learned directly in the feature space, enabling a seamless representation of complex structured domains.
We report on experimental results for real-world data sets showing that IGNs provide significant advances over state-of-the-art methods.
arXiv Detail & Related papers (2022-04-21T05:27:09Z) - Invariance Learning in Deep Neural Networks with Differentiable Laplace
Approximations [76.82124752950148]
We develop a convenient gradient-based method for selecting the data augmentation.
We use a differentiable Kronecker-factored Laplace approximation to the marginal likelihood as our objective.
arXiv Detail & Related papers (2022-02-22T02:51:11Z) - Consistency and Diversity induced Human Motion Segmentation [231.36289425663702]
We propose a novel Consistency and Diversity induced human Motion (CDMS) algorithm.
Our model factorizes the source and target data into distinct multi-layer feature spaces.
A multi-mutual learning strategy is carried out to reduce the domain gap between the source and target data.
arXiv Detail & Related papers (2022-02-10T06:23:56Z) - Functorial Manifold Learning [1.14219428942199]
We first characterize manifold learning algorithms as functors that map pseudometric spaces to optimization objectives.
We then use this characterization to prove refinement bounds on manifold learning loss functions and construct a hierarchy of manifold learning algorithms.
We express several popular manifold learning algorithms as functors at different levels of this hierarchy, including Metric Multidimensional Scaling, IsoMap, and UMAP.
arXiv Detail & Related papers (2020-11-15T02:30:23Z) - Extendable and invertible manifold learning with geometry regularized
autoencoders [9.742277703732187]
A fundamental task in data exploration is to extract simplified low dimensional representations that capture intrinsic geometry in data.
Common approaches to this task use kernel methods for manifold learning.
We present a new method for integrating both approaches by incorporating a geometric regularization term in the bottleneck of the autoencoder.
arXiv Detail & Related papers (2020-07-14T15:59:10Z) - A Trainable Optimal Transport Embedding for Feature Aggregation and its
Relationship to Attention [96.77554122595578]
We introduce a parametrized representation of fixed size, which embeds and then aggregates elements from a given input set according to the optimal transport plan between the set and a trainable reference.
Our approach scales to large datasets and allows end-to-end training of the reference, while also providing a simple unsupervised learning mechanism with small computational cost.
arXiv Detail & Related papers (2020-06-22T08:35:58Z) - Multi-Objective Genetic Programming for Manifold Learning: Balancing
Quality and Dimensionality [4.4181317696554325]
State-of-the-art manifold learning algorithms are opaque in how they perform this transformation.
We introduce a multi-objective approach that automatically balances the competing objectives of manifold quality and dimensionality.
Our proposed approach is competitive with a range of baseline and state-of-the-art manifold learning methods.
arXiv Detail & Related papers (2020-01-05T23:24:33Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.