Modern Non-Linear Function-on-Function Regression

• URL: http://arxiv.org/abs/2107.14151v2
• Date: Sat, 7 Oct 2023 17:57:39 GMT
• Title: Modern Non-Linear Function-on-Function Regression
• Authors: Aniruddha Rajendra Rao, Matthew Reimherr
• Abstract summary: We introduce a new class of non-linear function-on-function regression models for functional data using neural networks. We give two model fitting strategies, Functional Direct Neural Network (FDNN) and Functional Basis Neural Network (FBNN)
• Score: 8.231050911072755
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce a new class of non-linear function-on-function regression models for functional data using neural networks. We propose a framework using a hidden layer consisting of continuous neurons, called a continuous hidden layer, for functional response modeling and give two model fitting strategies, Functional Direct Neural Network (FDNN) and Functional Basis Neural Network (FBNN). Both are designed explicitly to exploit the structure inherent in functional data and capture the complex relations existing between the functional predictors and the functional response. We fit these models by deriving functional gradients and implement regularization techniques for more parsimonious results. We demonstrate the power and flexibility of our proposed method in handling complex functional models through extensive simulation studies as well as real data examples.

Related papers

• A Functional Extension of Semi-Structured Networks [2.482050942288848]
  Semi-structured networks (SSNs) merge structures familiar from additive models with deep neural networks. Inspired by large-scale datasets, this paper explores extending SSNs to functional data. We propose a functional SSN method that retains the advantageous properties of classical functional regression approaches while also improving scalability.
  arXiv  Detail & Related papers  (2024-10-07T18:50:18Z)
• Approximation of RKHS Functionals by Neural Networks [30.42446856477086]
  We study the approximation of functionals on kernel reproducing Hilbert spaces (RKHS's) using neural networks. We derive explicit error bounds for those induced by inverse multiquadric, Gaussian, and Sobolev kernels. We apply our findings to functional regression, proving that neural networks can accurately approximate the regression maps.
  arXiv  Detail & Related papers  (2024-03-18T18:58:23Z)
• Functional Neural Networks: Shift invariant models for functional data with applications to EEG classification [0.0]
  We introduce a new class of neural networks that are shift invariant and preserve smoothness of the data: functional neural networks (FNNs) For this, we use methods from functional data analysis (FDA) to extend multi-layer perceptrons and convolutional neural networks to functional data. We show that the models outperform a benchmark model from FDA in terms of accuracy and successfully use FNNs to classify electroencephalography (EEG) data.
  arXiv  Detail & Related papers  (2023-01-14T09:41:21Z)
• Neural Estimation of Submodular Functions with Applications to Differentiable Subset Selection [50.14730810124592]
  Submodular functions and variants, through their ability to characterize diversity and coverage, have emerged as a key tool for data selection and summarization. We propose FLEXSUBNET, a family of flexible neural models for both monotone and non-monotone submodular functions.
  arXiv  Detail & Related papers  (2022-10-20T06:00:45Z)
• Offline Reinforcement Learning with Differentiable Function Approximation is Provably Efficient [65.08966446962845]
  offline reinforcement learning, which aims at optimizing decision-making strategies with historical data, has been extensively applied in real-life applications. We take a step by considering offline reinforcement learning with differentiable function class approximation (DFA) Most importantly, we show offline differentiable function approximation is provably efficient by analyzing the pessimistic fitted Q-learning algorithm.
  arXiv  Detail & Related papers  (2022-10-03T07:59:42Z)
• Going Beyond Linear RL: Sample Efficient Neural Function Approximation [76.57464214864756]
  We study function approximation with two-layer neural networks. Our results significantly improve upon what can be attained with linear (or eluder dimension) methods.
  arXiv  Detail & Related papers  (2021-07-14T03:03:56Z)
• Non-linear Functional Modeling using Neural Networks [6.624726878647541]
  We introduce a new class of non-linear models for functional data based on neural networks. We propose two variations of our framework: a functional neural network with continuous hidden layers, and a second version that utilizes basis expansions and continuous hidden layers.
  arXiv  Detail & Related papers  (2021-04-19T14:59:55Z)
• UNIPoint: Universally Approximating Point Processes Intensities [125.08205865536577]
  We provide a proof that a class of learnable functions can universally approximate any valid intensity function. We implement UNIPoint, a novel neural point process model, using recurrent neural networks to parameterise sums of basis function upon each event.
  arXiv  Detail & Related papers  (2020-07-28T09:31:56Z)
• Provably Efficient Neural Estimation of Structural Equation Model: An Adversarial Approach [144.21892195917758]
  We study estimation in a class of generalized Structural equation models (SEMs) We formulate the linear operator equation as a min-max game, where both players are parameterized by neural networks (NNs), and learn the parameters of these neural networks using a gradient descent. For the first time we provide a tractable estimation procedure for SEMs based on NNs with provable convergence and without the need for sample splitting.
  arXiv  Detail & Related papers  (2020-07-02T17:55:47Z)
• Deep Learning with Functional Inputs [0.0]
  We present a methodology for integrating functional data into feed-forward neural networks. A by-product of the method is a set of dynamic functional weights that can be visualized during the optimization process. The model is shown to perform well in a number of contexts including prediction of new data and recovery of the true underlying functional weights.
  arXiv  Detail & Related papers  (2020-06-17T01:23:00Z)
• Measuring Model Complexity of Neural Networks with Curve Activation Functions [100.98319505253797]
  We propose the linear approximation neural network (LANN) to approximate a given deep model with curve activation function. We experimentally explore the training process of neural networks and detect overfitting. We find that the $L1$ and $L2$ regularizations suppress the increase of model complexity.
  arXiv  Detail & Related papers  (2020-06-16T07:38:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.