Cauchy activation function and XNet

• URL: http://arxiv.org/abs/2409.19221v1
• Date: Sat, 28 Sep 2024 03:25:33 GMT
• Title: Cauchy activation function and XNet
• Authors: Xin Li, Zhihong Xia, Hongkun Zhang,
• Abstract summary: We have developed a novel activation function, named the Cauchy Activation Function. This function is derived from the Cauchy Integral Theorem in complex analysis and is specifically tailored for problems requiring high precision. This innovation has led to the creation of a new class of neural networks, which we call (Comple)XNet, or simply XNet.
• Score: 3.9426000822656224
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We have developed a novel activation function, named the Cauchy Activation Function. This function is derived from the Cauchy Integral Theorem in complex analysis and is specifically tailored for problems requiring high precision. This innovation has led to the creation of a new class of neural networks, which we call (Comple)XNet, or simply XNet. We will demonstrate that XNet is particularly effective for high-dimensional challenges such as image classification and solving Partial Differential Equations (PDEs). Our evaluations show that XNet significantly outperforms established benchmarks like MNIST and CIFAR-10 in computer vision, and offers substantial advantages over Physics-Informed Neural Networks (PINNs) in both low-dimensional and high-dimensional PDE scenarios.

Related papers

• Enhancing Neural Function Approximation: The XNet Outperforming KAN [3.9426000822656224]
  XNet is a single-layer neural network architecture that leverages Cauchy integral-based activation functions for high-order function. We show that the Cauchy activation functions used in XNet can achieve arbitrary-order convergence. Results establish XNet as a highly efficient architecture for both scientific computing and AI applications.
  arXiv  Detail & Related papers  (2025-01-31T08:33:10Z)
• QuFeX: Quantum feature extraction module for hybrid quantum-classical deep neural networks [1.139490906109446]
  We introduce Quantum Feature Extraction (QuFeX), a novel quantum machine learning module. QuFeX enables feature extraction in a reduced-dimensional space, significantly decreasing the number of parallel evaluations required in typical quantum convolutional neural network architectures. As an application of QuFeX, we propose Qu-Net -- a hybrid architecture which integrates QuFeX at the bottleneck of a U-Net architecture.
  arXiv  Detail & Related papers  (2025-01-22T19:00:09Z)
• Task-Oriented Real-time Visual Inference for IoVT Systems: A Co-design Framework of Neural Networks and Edge Deployment [61.20689382879937]
  Task-oriented edge computing addresses this by shifting data analysis to the edge. Existing methods struggle to balance high model performance with low resource consumption. We propose a novel co-design framework to optimize neural network architecture.
  arXiv  Detail & Related papers  (2024-10-29T19:02:54Z)
• Model Comparisons: XNet Outperforms KAN [3.9426000822656224]
  XNet is a novel algorithm that employs the complex-valued Cauchy integral formula. XNet significant improves speed and accuracy across various tasks in both low and high-dimensional spaces.
  arXiv  Detail & Related papers  (2024-10-02T20:59:47Z)
• A Novel Paradigm for Neural Computation: X-Net with Learnable Neurons and Adaptable Structure [29.11456970277094]
  We show that X-Net can achieve comparable or even better performance than neurons on regression and classification tasks. X-Net is shown to help scientists discover new laws of mathematics or physics.
  arXiv  Detail & Related papers  (2024-01-03T14:52:18Z)
• GIT-Net: Generalized Integral Transform for Operator Learning [58.13313857603536]
  This article introduces GIT-Net, a deep neural network architecture for approximating Partial Differential Equation (PDE) operators. GIT-Net harnesses the fact that differential operators commonly used for defining PDEs can often be represented parsimoniously when expressed in specialized functional bases. Numerical experiments demonstrate that GIT-Net is a competitive neural network operator, exhibiting small test errors and low evaluations across a range of PDE problems.
  arXiv  Detail & Related papers  (2023-12-05T03:03:54Z)
• KXNet: A Model-Driven Deep Neural Network for Blind Super-Resolution [57.882146858582175]
  We propose a model-driven deep neural network, called KXNet, for blind SISR. The proposed KXNet is fully integrated with the inherent physical mechanism underlying this SISR task. Experiments on synthetic and real data finely demonstrate the superior accuracy and generality of our method.
  arXiv  Detail & Related papers  (2022-09-21T12:22:50Z)
• Benefits of Overparameterized Convolutional Residual Networks: Function Approximation under Smoothness Constraint [48.25573695787407]
  We prove that large ConvResNets can not only approximate a target function in terms of function value, but also exhibit sufficient first-order smoothness. Our theory partially justifies the benefits of using deep and wide networks in practice.
  arXiv  Detail & Related papers  (2022-06-09T15:35:22Z)
• FxP-QNet: A Post-Training Quantizer for the Design of Mixed Low-Precision DNNs with Dynamic Fixed-Point Representation [2.4149105714758545]
  We propose a novel framework referred to as the Fixed-Point Quantizer of deep neural Networks (FxP-QNet) FxP-QNet adapts the quantization level for each data-structure of each layer based on the trade-off between the network accuracy and the low-precision requirements. Results show that FxP-QNet-quantized AlexNet, VGG-16, and ResNet-18 reduce the overall memory requirements of their full-precision counterparts by 7.16x, 10.36x, and 6.44x with less than 0.95%, 0.95%, and 1.99%
  arXiv  Detail & Related papers  (2022-03-22T23:01:43Z)
• CDiNN -Convex Difference Neural Networks [0.8122270502556374]
  Neural networks with ReLU activation function have been shown to be universal function approximators learn function mapping as non-smooth functions. New neural network architecture called ICNNs learn the output as a convex input.
  arXiv  Detail & Related papers  (2021-03-31T17:31:16Z)
• Modeling from Features: a Mean-field Framework for Over-parameterized Deep Neural Networks [54.27962244835622]
  This paper proposes a new mean-field framework for over- parameterized deep neural networks (DNNs) In this framework, a DNN is represented by probability measures and functions over its features in the continuous limit. We illustrate the framework via the standard DNN and the Residual Network (Res-Net) architectures.
  arXiv  Detail & Related papers  (2020-07-03T01:37:16Z)
• Widening and Squeezing: Towards Accurate and Efficient QNNs [125.172220129257]
  Quantization neural networks (QNNs) are very attractive to the industry because their extremely cheap calculation and storage overhead, but their performance is still worse than that of networks with full-precision parameters. Most of existing methods aim to enhance performance of QNNs especially binary neural networks by exploiting more effective training techniques. We address this problem by projecting features in original full-precision networks to high-dimensional quantization features.
  arXiv  Detail & Related papers  (2020-02-03T04:11:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.