Functional Autoencoder for Smoothing and Representation Learning
- URL: http://arxiv.org/abs/2401.09499v1
- Date: Wed, 17 Jan 2024 08:33:25 GMT
- Title: Functional Autoencoder for Smoothing and Representation Learning
- Authors: Sidi Wu, C\'edric Beaulac and Jiguo Cao
- Abstract summary: We propose to learn the nonlinear representations of functional data using neural network autoencoders designed to process data in the form it is usually collected without the need of preprocessing.
We design the encoder to employ a projection layer computing the weighted inner product of the functional data and functional weights over the observed timestamp, and the decoder to apply a recovery layer that maps the finite-dimensional vector extracted from the functional data back to functional space.
- Score: 0.0
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: A common pipeline in functional data analysis is to first convert the
discretely observed data to smooth functions, and then represent the functions
by a finite-dimensional vector of coefficients summarizing the information.
Existing methods for data smoothing and dimensional reduction mainly focus on
learning the linear mappings from the data space to the representation space,
however, learning only the linear representations may not be sufficient. In
this study, we propose to learn the nonlinear representations of functional
data using neural network autoencoders designed to process data in the form it
is usually collected without the need of preprocessing. We design the encoder
to employ a projection layer computing the weighted inner product of the
functional data and functional weights over the observed timestamp, and the
decoder to apply a recovery layer that maps the finite-dimensional vector
extracted from the functional data back to functional space using a set of
predetermined basis functions. The developed architecture can accommodate both
regularly and irregularly spaced data. Our experiments demonstrate that the
proposed method outperforms functional principal component analysis in terms of
prediction and classification, and maintains superior smoothing ability and
better computational efficiency in comparison to the conventional autoencoders
under both linear and nonlinear settings.
Related papers
- Nonlinear functional regression by functional deep neural network with
kernel embedding [20.306390874610635]
We propose a functional deep neural network with an efficient and fully data-dependent dimension reduction method.
The architecture of our functional net consists of a kernel embedding step, a projection step, and a deep ReLU neural network for the prediction.
The utilization of smooth kernel embedding enables our functional net to be discretization invariant, efficient, and robust to noisy observations.
arXiv Detail & Related papers (2024-01-05T16:43:39Z) - A Functional approach for Two Way Dimension Reduction in Time Series [13.767812547998735]
We propose a non-linear function-on-function approach, which consists of a functional encoder and a functional decoder.
Our approach gives a low dimension latent representation by reducing the number of functional features as well as the timepoints at which the functions are observed.
arXiv Detail & Related papers (2023-01-01T06:09:15Z) - Towards a mathematical understanding of learning from few examples with
nonlinear feature maps [68.8204255655161]
We consider the problem of data classification where the training set consists of just a few data points.
We reveal key relationships between the geometry of an AI model's feature space, the structure of the underlying data distributions, and the model's generalisation capabilities.
arXiv Detail & Related papers (2022-11-07T14:52:58Z) - Functional Nonlinear Learning [0.0]
We propose a functional nonlinear learning (FunNoL) method to represent multivariate functional data in a lower-dimensional feature space.
We show that FunNoL provides satisfactory curve classification and reconstruction regardless of data sparsity.
arXiv Detail & Related papers (2022-06-22T23:47:45Z) - NOMAD: Nonlinear Manifold Decoders for Operator Learning [17.812064311297117]
Supervised learning in function spaces is an emerging area of machine learning research.
We show NOMAD, a novel operator learning framework with a nonlinear decoder map capable of learning finite dimensional representations of nonlinear submanifolds in function spaces.
arXiv Detail & Related papers (2022-06-07T19:52:44Z) - Towards Open-World Feature Extrapolation: An Inductive Graph Learning
Approach [80.8446673089281]
We propose a new learning paradigm with graph representation and learning.
Our framework contains two modules: 1) a backbone network (e.g., feedforward neural nets) as a lower model takes features as input and outputs predicted labels; 2) a graph neural network as an upper model learns to extrapolate embeddings for new features via message passing over a feature-data graph built from observed data.
arXiv Detail & Related papers (2021-10-09T09:02:45Z) - Efficient Multidimensional Functional Data Analysis Using Marginal
Product Basis Systems [2.4554686192257424]
We propose a framework for learning continuous representations from a sample of multidimensional functional data.
We show that the resulting estimation problem can be solved efficiently by the tensor decomposition.
We conclude with a real data application in neuroimaging.
arXiv Detail & Related papers (2021-07-30T16:02:15Z) - Rank-R FNN: A Tensor-Based Learning Model for High-Order Data
Classification [69.26747803963907]
Rank-R Feedforward Neural Network (FNN) is a tensor-based nonlinear learning model that imposes Canonical/Polyadic decomposition on its parameters.
First, it handles inputs as multilinear arrays, bypassing the need for vectorization, and can thus fully exploit the structural information along every data dimension.
We establish the universal approximation and learnability properties of Rank-R FNN, and we validate its performance on real-world hyperspectral datasets.
arXiv Detail & Related papers (2021-04-11T16:37:32Z) - Representation Learning for Sequence Data with Deep Autoencoding
Predictive Components [96.42805872177067]
We propose a self-supervised representation learning method for sequence data, based on the intuition that useful representations of sequence data should exhibit a simple structure in the latent space.
We encourage this latent structure by maximizing an estimate of predictive information of latent feature sequences, which is the mutual information between past and future windows at each time step.
We demonstrate that our method recovers the latent space of noisy dynamical systems, extracts predictive features for forecasting tasks, and improves automatic speech recognition when used to pretrain the encoder on large amounts of unlabeled data.
arXiv Detail & Related papers (2020-10-07T03:34:01Z) - MetaSDF: Meta-learning Signed Distance Functions [85.81290552559817]
Generalizing across shapes with neural implicit representations amounts to learning priors over the respective function space.
We formalize learning of a shape space as a meta-learning problem and leverage gradient-based meta-learning algorithms to solve this task.
arXiv Detail & Related papers (2020-06-17T05:14:53Z) - Invariant Feature Coding using Tensor Product Representation [75.62232699377877]
We prove that the group-invariant feature vector contains sufficient discriminative information when learning a linear classifier.
A novel feature model that explicitly consider group action is proposed for principal component analysis and k-means clustering.
arXiv Detail & Related papers (2019-06-05T07:15:17Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.