An Informal Introduction to Multiplet Neural Networks
- URL: http://arxiv.org/abs/2006.01606v1
- Date: Tue, 2 Jun 2020 13:46:32 GMT
- Title: An Informal Introduction to Multiplet Neural Networks
- Authors: Nathan E. Frick
- Abstract summary: I replace the dot product with the weighted Lehmer mean, which may emulate different cases of a generalized mean.
The generalization parameter is typically set to a different value for each neuron in the multiplet.
Some properties of the network are investigated, showing the capacity to emulate the classical exclusive-or problem in two layers.
- Score: 0.0
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In the artificial neuron, I replace the dot product with the weighted Lehmer
mean, which may emulate different cases of a generalized mean. The single
neuron instance is replaced by a multiplet of neurons which have the same
averaging weights. A group of outputs feed forward, in lieu of the single
scalar. The generalization parameter is typically set to a different value for
each neuron in the multiplet.
I further extend the concept to a multiplet taken from the Gini mean.
Derivatives with respect to the weight parameters and with respect to the two
generalization parameters are given.
Some properties of the network are investigated, showing the capacity to
emulate the classical exclusive-or problem organically in two layers and
perform some multiplication and division. The network can instantiate truncated
power series and variants, which can be used to approximate different
functions, provided that parameters are constrained.
Moreover, a mean case slope score is derived that can facilitate a
learning-rate novelty based on homogeneity of the selected elements. The
multiplet neuron equation provides a way to segment regularization timeframes
and approaches.
Related papers
- No One-Size-Fits-All Neurons: Task-based Neurons for Artificial Neural Networks [25.30801109401654]
Since the human brain is a task-based neuron user, can the artificial network design go from the task-based architecture design to the task-based neuron design?
We propose a two-step framework for prototyping task-based neurons.
Experiments show that the proposed task-based neuron design is not only feasible but also delivers competitive performance over other state-of-the-art models.
arXiv Detail & Related papers (2024-05-03T09:12:46Z) - Soft Matching Distance: A metric on neural representations that captures
single-neuron tuning [6.5714523708869566]
Common measures of neural representational (dis)similarity are designed to be insensitive to rotations and reflections of the neural activation space.
We propose a new metric to measure distances between networks with different sizes.
arXiv Detail & Related papers (2023-11-16T00:13:00Z) - Multilayer Multiset Neuronal Networks -- MMNNs [55.2480439325792]
The present work describes multilayer multiset neuronal networks incorporating two or more layers of coincidence similarity neurons.
The work also explores the utilization of counter-prototype points, which are assigned to the image regions to be avoided.
arXiv Detail & Related papers (2023-08-28T12:55:13Z) - Cones: Concept Neurons in Diffusion Models for Customized Generation [41.212255848052514]
This paper finds a small cluster of neurons in a diffusion model corresponding to a particular subject.
The concept neurons demonstrate magnetic properties in interpreting and manipulating generation results.
For large-scale applications, the concept neurons are environmentally friendly as we only need to store a sparse cluster of int index instead of dense float32 values.
arXiv Detail & Related papers (2023-03-09T09:16:04Z) - Permutation Equivariant Neural Functionals [92.0667671999604]
This work studies the design of neural networks that can process the weights or gradients of other neural networks.
We focus on the permutation symmetries that arise in the weights of deep feedforward networks because hidden layer neurons have no inherent order.
In our experiments, we find that permutation equivariant neural functionals are effective on a diverse set of tasks.
arXiv Detail & Related papers (2023-02-27T18:52:38Z) - The activity-weight duality in feed forward neural networks: The
geometric determinants of generalization [7.372592187197655]
We find an exact duality between changes in activities in a given layer of neurons and changes in weights that connect to the next layer of neurons in a densely connected layer in any feed forward neural network.
These insights can be used to guide development of algorithms for finding more generalizable solutions in overparametrized neural networks.
arXiv Detail & Related papers (2022-03-21T05:00:54Z) - The Separation Capacity of Random Neural Networks [78.25060223808936]
We show that a sufficiently large two-layer ReLU-network with standard Gaussian weights and uniformly distributed biases can solve this problem with high probability.
We quantify the relevant structure of the data in terms of a novel notion of mutual complexity.
arXiv Detail & Related papers (2021-07-31T10:25:26Z) - On the training of sparse and dense deep neural networks: less
parameters, same performance [0.0]
We propose a variant of the spectral learning method as appeared in Giambagli et al Nat. Comm. 2021.
The eigenvalues act as veritable knobs which can be freely tuned so as to (i) enhance, or alternatively silence, the contribution of the input nodes.
Each spectral parameter reflects back on the whole set of inter-nodes weights, an attribute which we shall effectively exploit to yield sparse networks with stunning classification abilities.
arXiv Detail & Related papers (2021-06-17T14:54:23Z) - Provably Efficient Neural Estimation of Structural Equation Model: An
Adversarial Approach [144.21892195917758]
We study estimation in a class of generalized Structural equation models (SEMs)
We formulate the linear operator equation as a min-max game, where both players are parameterized by neural networks (NNs), and learn the parameters of these neural networks using a gradient descent.
For the first time we provide a tractable estimation procedure for SEMs based on NNs with provable convergence and without the need for sample splitting.
arXiv Detail & Related papers (2020-07-02T17:55:47Z) - Multipole Graph Neural Operator for Parametric Partial Differential
Equations [57.90284928158383]
One of the main challenges in using deep learning-based methods for simulating physical systems is formulating physics-based data.
We propose a novel multi-level graph neural network framework that captures interaction at all ranges with only linear complexity.
Experiments confirm our multi-graph network learns discretization-invariant solution operators to PDEs and can be evaluated in linear time.
arXiv Detail & Related papers (2020-06-16T21:56:22Z) - Generalizing Convolutional Neural Networks for Equivariance to Lie
Groups on Arbitrary Continuous Data [52.78581260260455]
We propose a general method to construct a convolutional layer that is equivariant to transformations from any specified Lie group.
We apply the same model architecture to images, ball-and-stick molecular data, and Hamiltonian dynamical systems.
arXiv Detail & Related papers (2020-02-25T17:40:38Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.