Ladder Polynomial Neural Networks

• URL: http://arxiv.org/abs/2106.13834v2
• Date: Tue, 29 Jun 2021 04:57:17 GMT
• Title: Ladder Polynomial Neural Networks
• Authors: Li-Ping Liu, Ruiyuan Gu, Xiaozhe Hu
• Abstract summary: Polynomial functions have plenty of useful analytical properties, but they are rarely used as learning models because their function class is considered to be restricted. This work constructs feedforward neural networks using the product activation, a new activation function constructed from multiplications.
• Score: 6.902168821854859
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Polynomial functions have plenty of useful analytical properties, but they are rarely used as learning models because their function class is considered to be restricted. This work shows that when trained properly polynomial functions can be strong learning models. Particularly this work constructs polynomial feedforward neural networks using the product activation, a new activation function constructed from multiplications. The new neural network is a polynomial function and provides accurate control of its polynomial order. It can be trained by standard training techniques such as batch normalization and dropout. This new feedforward network covers several previous polynomial models as special cases. Compared with common feedforward neural networks, the polynomial feedforward network has closed-form calculations of a few interesting quantities, which are very useful in Bayesian learning. In a series of regression and classification tasks in the empirical study, the proposed model outperforms previous polynomial models.

Related papers

• Polynomial, trigonometric, and tropical activations [1.534667887016089]
  This article explores families of functions based on orthonormal bases, including the Hermite basis and the trigonometric basis.<n>We show that through simple variance-preserving and without additional clamping mechanisms, these activations can successfully be used to train deep models.
  arXiv  Detail & Related papers  (2025-02-03T11:13:58Z)
• Multilinear Operator Networks [60.7432588386185]
  Polynomial Networks is a class of models that does not require activation functions. We propose MONet, which relies solely on multilinear operators.
  arXiv  Detail & Related papers  (2024-01-31T16:52:19Z)
• Regularization of polynomial networks for image recognition [78.4786845859205]
  Polynomial Networks (PNs) have emerged as an alternative method with a promising performance and improved interpretability. We introduce a class of PNs, which are able to reach the performance of ResNet across a range of six benchmarks.
  arXiv  Detail & Related papers  (2023-03-24T10:05:22Z)
• A Tutorial on Neural Networks and Gradient-free Training [0.0]
  This paper presents a compact, matrix-based representation of neural networks in a self-contained tutorial fashion. neural networks are mathematical nonlinear functions constructed by composing several vector-valued functions.
  arXiv  Detail & Related papers  (2022-11-26T15:33:11Z)
• Neural Estimation of Submodular Functions with Applications to Differentiable Subset Selection [50.14730810124592]
  Submodular functions and variants, through their ability to characterize diversity and coverage, have emerged as a key tool for data selection and summarization. We propose FLEXSUBNET, a family of flexible neural models for both monotone and non-monotone submodular functions.
  arXiv  Detail & Related papers  (2022-10-20T06:00:45Z)
• Interaction Decompositions for Tensor Network Regression [0.0]
  We show how to assess the relative importance of different regressors as a function of their degree. We introduce a new type of tensor network model that is explicitly trained on only a small subset of interaction degrees. This suggests that standard tensor network models utilize their regressors in an inefficient manner, with the lower degree terms vastly underutilized.
  arXiv  Detail & Related papers  (2022-08-11T20:17:27Z)
• Bagged Polynomial Regression and Neural Networks [0.0]
  Series and dataset regression are able to approximate the same function classes as neural networks. textitbagged regression (BPR) is an attractive alternative to neural networks. BPR performs as well as neural networks in crop classification using satellite data.
  arXiv  Detail & Related papers  (2022-05-17T19:55:56Z)
• NN2Poly: A polynomial representation for deep feed-forward artificial neural networks [0.6502001911298337]
  NN2Poly is a theoretical approach to obtain an explicit model of an already trained fully-connected feed-forward artificial neural network. This approach extends a previous idea proposed in the literature, which was limited to single hidden layer networks.
  arXiv  Detail & Related papers  (2021-12-21T17:55:22Z)
• Gone Fishing: Neural Active Learning with Fisher Embeddings [55.08537975896764]
  There is an increasing need for active learning algorithms that are compatible with deep neural networks. This article introduces BAIT, a practical representation of tractable, and high-performing active learning algorithm for neural networks.
  arXiv  Detail & Related papers  (2021-06-17T17:26:31Z)
• Towards a mathematical framework to inform Neural Network modelling via Polynomial Regression [0.0]
  It is shown that almost identical predictions can be made when certain conditions are met locally. When learning from generated data, the proposed method producess that approximate correctly the data locally.
  arXiv  Detail & Related papers  (2021-02-07T17:56:16Z)
• On Function Approximation in Reinforcement Learning: Optimism in the Face of Large State Spaces [208.67848059021915]
  We study the exploration-exploitation tradeoff at the core of reinforcement learning. In particular, we prove that the complexity of the function class $mathcalF$ characterizes the complexity of the function. Our regret bounds are independent of the number of episodes.
  arXiv  Detail & Related papers  (2020-11-09T18:32:22Z)
• Deep Polynomial Neural Networks [77.70761658507507]
  $Pi$Nets are a new class of function approximators based on expansions. $Pi$Nets produce state-the-art results in three challenging tasks, i.e. image generation, face verification and 3D mesh representation learning.
  arXiv  Detail & Related papers  (2020-06-20T16:23:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.