Deep Recursive Embedding for High-Dimensional Data
- URL: http://arxiv.org/abs/2104.05171v1
- Date: Mon, 12 Apr 2021 03:04:38 GMT
- Title: Deep Recursive Embedding for High-Dimensional Data
- Authors: Zixia Zhou, Yuanyuan Wang, Boudewijn P.F. Lelieveldt, Qian Tao
- Abstract summary: We propose to combine the deep neural network (DNN) with the mathematical-grounded embedding rules for high-dimensional data embedding.
Our experiments demonstrated the excellent performance of the proposed Deep Recursive Embedding (DRE) on high-dimensional data embedding.
- Score: 10.499461691493526
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: t-distributed stochastic neighbor embedding (t-SNE) is a well-established
visualization method for complex high-dimensional data. However, the original
t-SNE method is nonparametric, stochastic, and often cannot well prevserve the
global structure of data as it emphasizes local neighborhood. With t-SNE as a
reference, we propose to combine the deep neural network (DNN) with the
mathematical-grounded embedding rules for high-dimensional data embedding. We
first introduce a deep embedding network (DEN) framework, which can learn a
parametric mapping from high-dimensional space to low-dimensional embedding.
DEN has a flexible architecture that can accommodate different input data
(vector, image, or tensor) and loss functions. To improve the embedding
performance, a recursive training strategy is proposed to make use of the
latent representations extracted by DEN. Finally, we propose a two-stage loss
function combining the advantages of two popular embedding methods, namely,
t-SNE and uniform manifold approximation and projection (UMAP), for optimal
visualization effect. We name the proposed method Deep Recursive Embedding
(DRE), which optimizes DEN with a recursive training strategy and two-stage
losse. Our experiments demonstrated the excellent performance of the proposed
DRE method on high-dimensional data embedding, across a variety of public
databases. Remarkably, our comparative results suggested that our proposed DRE
could lead to improved global structure preservation.
Related papers
- Hyperboloid GPLVM for Discovering Continuous Hierarchies via Nonparametric Estimation [41.13597666007784]
Dimensionality reduction (DR) offers a useful representation of complex high-dimensional data.
Recent DR methods focus on hyperbolic geometry to derive a faithful low-dimensional representation of hierarchical data.
This paper presents hGP-LVMs to embed high-dimensional hierarchical data with implicit continuity via nonparametric estimation.
arXiv Detail & Related papers (2024-10-22T05:07:30Z) - Laplacian-based Cluster-Contractive t-SNE for High Dimensional Data
Visualization [20.43471678277403]
We propose LaptSNE, a new graph-based dimensionality reduction method based on t-SNE.
Specifically, LaptSNE leverages the eigenvalue information of the graph Laplacian to shrink the potential clusters in the low-dimensional embedding.
We show how to calculate the gradient analytically, which may be of broad interest when considering optimization with Laplacian-composited objective.
arXiv Detail & Related papers (2022-07-25T14:10:24Z) - Optimization-Based Separations for Neural Networks [57.875347246373956]
We show that gradient descent can efficiently learn ball indicator functions using a depth 2 neural network with two layers of sigmoidal activations.
This is the first optimization-based separation result where the approximation benefits of the stronger architecture provably manifest in practice.
arXiv Detail & Related papers (2021-12-04T18:07:47Z) - Deep Recursive Embedding for High-Dimensional Data [9.611123249318126]
We propose to combine deep neural networks (DNN) with mathematics-guided embedding rules for high-dimensional data embedding.
We introduce a generic deep embedding network (DEN) framework, which is able to learn a parametric mapping from high-dimensional space to low-dimensional space.
arXiv Detail & Related papers (2021-10-31T23:22:33Z) - Index $t$-SNE: Tracking Dynamics of High-Dimensional Datasets with
Coherent Embeddings [1.7188280334580195]
This paper presents a methodology to reuse an embedding to create a new one, where cluster positions are preserved.
The proposed algorithm has the same complexity as the original $t$-SNE to embed new items, and a lower one when considering the embedding of a dataset sliced into sub-pieces.
arXiv Detail & Related papers (2021-09-22T06:45:37Z) - Improving Metric Dimensionality Reduction with Distributed Topology [68.8204255655161]
DIPOLE is a dimensionality-reduction post-processing step that corrects an initial embedding by minimizing a loss functional with both a local, metric term and a global, topological term.
We observe that DIPOLE outperforms popular methods like UMAP, t-SNE, and Isomap on a number of popular datasets.
arXiv Detail & Related papers (2021-06-14T17:19:44Z) - Rank-R FNN: A Tensor-Based Learning Model for High-Order Data
Classification [69.26747803963907]
Rank-R Feedforward Neural Network (FNN) is a tensor-based nonlinear learning model that imposes Canonical/Polyadic decomposition on its parameters.
First, it handles inputs as multilinear arrays, bypassing the need for vectorization, and can thus fully exploit the structural information along every data dimension.
We establish the universal approximation and learnability properties of Rank-R FNN, and we validate its performance on real-world hyperspectral datasets.
arXiv Detail & Related papers (2021-04-11T16:37:32Z) - Densely Nested Top-Down Flows for Salient Object Detection [137.74130900326833]
This paper revisits the role of top-down modeling in salient object detection.
It designs a novel densely nested top-down flows (DNTDF)-based framework.
In every stage of DNTDF, features from higher levels are read in via the progressive compression shortcut paths (PCSP)
arXiv Detail & Related papers (2021-02-18T03:14:02Z) - Dual-constrained Deep Semi-Supervised Coupled Factorization Network with
Enriched Prior [80.5637175255349]
We propose a new enriched prior based Dual-constrained Deep Semi-Supervised Coupled Factorization Network, called DS2CF-Net.
To ex-tract hidden deep features, DS2CF-Net is modeled as a deep-structure and geometrical structure-constrained neural network.
Our network can obtain state-of-the-art performance for representation learning and clustering.
arXiv Detail & Related papers (2020-09-08T13:10:21Z) - Two-Dimensional Semi-Nonnegative Matrix Factorization for Clustering [50.43424130281065]
We propose a new Semi-Nonnegative Matrix Factorization method for 2-dimensional (2D) data, named TS-NMF.
It overcomes the drawback of existing methods that seriously damage the spatial information of the data by converting 2D data to vectors in a preprocessing step.
arXiv Detail & Related papers (2020-05-19T05:54:14Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.