Exploring Data Geometry for Continual Learning
- URL: http://arxiv.org/abs/2304.03931v1
- Date: Sat, 8 Apr 2023 06:35:25 GMT
- Title: Exploring Data Geometry for Continual Learning
- Authors: Zhi Gao, Chen Xu, Feng Li, Yunde Jia, Mehrtash Harandi, Yuwei Wu
- Abstract summary: We study continual learning from a novel perspective by exploring data geometry for the non-stationary stream of data.
Our method dynamically expands the geometry of the underlying space to match growing geometric structures induced by new data.
Experiments show that our method achieves better performance than baseline methods designed in Euclidean space.
- Score: 64.4358878435983
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Continual learning aims to efficiently learn from a non-stationary stream of
data while avoiding forgetting the knowledge of old data. In many practical
applications, data complies with non-Euclidean geometry. As such, the commonly
used Euclidean space cannot gracefully capture non-Euclidean geometric
structures of data, leading to inferior results. In this paper, we study
continual learning from a novel perspective by exploring data geometry for the
non-stationary stream of data. Our method dynamically expands the geometry of
the underlying space to match growing geometric structures induced by new data,
and prevents forgetting by keeping geometric structures of old data into
account. In doing so, making use of the mixed curvature space, we propose an
incremental search scheme, through which the growing geometric structures are
encoded. Then, we introduce an angular-regularization loss and a
neighbor-robustness loss to train the model, capable of penalizing the change
of global geometric structures and local geometric structures. Experiments show
that our method achieves better performance than baseline methods designed in
Euclidean space.
Related papers
- Geometry-Aware Generative Autoencoders for Warped Riemannian Metric Learning and Generative Modeling on Data Manifolds [18.156807299614503]
We introduce Geometry-Aware Generative Autoencoder (GAGA), a novel framework that combines manifold learning with generative modeling.
GAGA shows competitive performance in simulated and real-world datasets, including a 30% improvement over the state-of-the-art methods in single-cell population-level trajectory inference.
arXiv Detail & Related papers (2024-10-16T17:53:26Z) - (Deep) Generative Geodesics [57.635187092922976]
We introduce a newian metric to assess the similarity between any two data points.
Our metric leads to the conceptual definition of generative distances and generative geodesics.
Their approximations are proven to converge to their true values under mild conditions.
arXiv Detail & Related papers (2024-07-15T21:14:02Z) - Beyond Euclid: An Illustrated Guide to Modern Machine Learning with Geometric, Topological, and Algebraic Structures [3.5357133304100326]
Modern machine learning encounters richly structured data that is inherently nonEuclidean.
Extracting knowledge from such non-Euclidean data requires a broader mathematical perspective.
We propose a graphical taxonomy that integrates recent advances into an intuitive unified framework.
arXiv Detail & Related papers (2024-07-12T17:48:36Z) - A Survey of Geometric Graph Neural Networks: Data Structures, Models and
Applications [67.33002207179923]
This paper presents a survey of data structures, models, and applications related to geometric GNNs.
We provide a unified view of existing models from the geometric message passing perspective.
We also summarize the applications as well as the related datasets to facilitate later research for methodology development and experimental evaluation.
arXiv Detail & Related papers (2024-03-01T12:13:04Z) - Adaptive Surface Normal Constraint for Geometric Estimation from Monocular Images [56.86175251327466]
We introduce a novel approach to learn geometries such as depth and surface normal from images while incorporating geometric context.
Our approach extracts geometric context that encodes the geometric variations present in the input image and correlates depth estimation with geometric constraints.
Our method unifies depth and surface normal estimations within a cohesive framework, which enables the generation of high-quality 3D geometry from images.
arXiv Detail & Related papers (2024-02-08T17:57:59Z) - Study of Manifold Geometry using Multiscale Non-Negative Kernel Graphs [32.40622753355266]
We propose a framework to study the geometric structure of the data.
We make use of our recently introduced non-negative kernel (NNK) regression graphs to estimate the point density, intrinsic dimension, and the linearity of the data manifold (curvature)
arXiv Detail & Related papers (2022-10-31T17:01:17Z) - Curved Geometric Networks for Visual Anomaly Recognition [39.91252195360767]
Learning a latent embedding to understand the underlying nature of data distribution is often formulated in Euclidean spaces with zero curvature.
In this work, we investigate benefits of the curved space for analyzing anomalies or out-of-distribution objects in data.
arXiv Detail & Related papers (2022-08-02T01:15:39Z) - Quadric hypersurface intersection for manifold learning in feature space [52.83976795260532]
manifold learning technique suitable for moderately high dimension and large datasets.
The technique is learned from the training data in the form of an intersection of quadric hypersurfaces.
At test time, this manifold can be used to introduce an outlier score for arbitrary new points.
arXiv Detail & Related papers (2021-02-11T18:52:08Z) - Computational Analysis of Deformable Manifolds: from Geometric Modelling
to Deep Learning [0.0]
We will show that the diversity of non-flat spaces provides a rich area of study.
We will explore geometric methods for shape processing and data analysis.
arXiv Detail & Related papers (2020-09-03T16:50:48Z) - DSG-Net: Learning Disentangled Structure and Geometry for 3D Shape
Generation [98.96086261213578]
We introduce DSG-Net, a deep neural network that learns a disentangled structured and geometric mesh representation for 3D shapes.
This supports a range of novel shape generation applications with disentangled control, such as of structure (geometry) while keeping geometry (structure) unchanged.
Our method not only supports controllable generation applications but also produces high-quality synthesized shapes, outperforming state-of-the-art methods.
arXiv Detail & Related papers (2020-08-12T17:06:51Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.