Convolutional Kolmogorov-Arnold Networks

• URL: http://arxiv.org/abs/2406.13155v2
• Date: Mon, 04 Nov 2024 00:55:06 GMT
• Title: Convolutional Kolmogorov-Arnold Networks
• Authors: Alexander Dylan Bodner, Antonio Santiago Tepsich, Jack Natan Spolski, Santiago Pourteau,
• Abstract summary: We introduce Convolutional Kolmogorov-Arnold Networks (Convolutional KANs) In this paper, we empirically validate the performance of Convolutional KANs against traditional architectures across Fashion-MNIST dataset. Experiments show that KAN Convolutions seem to learn more per kernel, which opens up a new horizon of possibilities in deep learning for computer vision.
• Score: 41.94295877935867
• License:
• Abstract: In this paper, we introduce Convolutional Kolmogorov-Arnold Networks (Convolutional KANs), an innovative alternative to the standard Convolutional Neural Networks (CNNs) that have revolutionized the field of computer vision. By integrating the learneable non-linear activation functions presented in Kolmogorov-Arnold Networks (KANs) into convolutions, we propose a new layer. Throughout the paper, we empirically validate the performance of Convolutional KANs against traditional architectures across Fashion-MNIST dataset, finding that, in some cases, this new approach maintains a similar level of accuracy while using half the number of parameters. This experiments show that KAN Convolutions seem to learn more per kernel, which opens up a new horizon of possibilities in deep learning for computer vision.

Related papers

• Residual Kolmogorov-Arnold Network for Enhanced Deep Learning [0.5852077003870417]
  We introduce Residual Arnold, which incorporates the Kolmogorov-KAN framework as a residual component. Our results demonstrate the potential of RKAN to enhance the capabilities of deep CNNs in visual data.
  arXiv  Detail & Related papers  (2024-10-07T21:12:32Z)
• Kolmogorov-Arnold Network Autoencoders [0.0]
  Kolmogorov-Arnold Networks (KANs) are promising alternatives to Multi-Layer Perceptrons (MLPs) KANs align closely with the Kolmogorov-Arnold representation theorem, potentially enhancing both model accuracy and interpretability. Our results demonstrate that KAN-based autoencoders achieve competitive performance in terms of reconstruction accuracy.
  arXiv  Detail & Related papers  (2024-10-02T22:56:00Z)
• Kolmogorov-Arnold Convolutions: Design Principles and Empirical Studies [0.0]
  This paper explores the application of Kolmogorov-Arnold Networks (KANs) in the domain of computer vision (CV) We propose a parameter-efficient design for Kolmogorov-Arnold convolutional layers and a parameter-efficient finetuning algorithm for pre-trained KAN models. We provide empirical evaluations conducted on MNIST, CIFAR10, CIFAR100, Tiny ImageNet, ImageNet1k, and HAM10000 datasets for image classification tasks.
  arXiv  Detail & Related papers  (2024-07-01T08:49:33Z)
• Mechanism of feature learning in convolutional neural networks [14.612673151889615]
  We identify the mechanism of how convolutional neural networks learn from image data. We present empirical evidence for our ansatz, including identifying high correlation between covariances of filters and patch-based AGOPs. We then demonstrate the generality of our result by using the patch-based AGOP to enable deep feature learning in convolutional kernel machines.
  arXiv  Detail & Related papers  (2023-09-01T16:30:02Z)
• From NeurODEs to AutoencODEs: a mean-field control framework for width-varying Neural Networks [68.8204255655161]
  We propose a new type of continuous-time control system, called AutoencODE, based on a controlled field that drives dynamics. We show that many architectures can be recovered in regions where the loss function is locally convex.
  arXiv  Detail & Related papers  (2023-07-05T13:26:17Z)
• Combining Neuro-Evolution of Augmenting Topologies with Convolutional Neural Networks [0.0]
  We combine NeuroEvolution of Augmenting Topologies (NEAT) with Convolutional Neural Networks (CNNs) and propose such a system using blocks of Residual Networks (ResNets) We explain how our suggested system can only be built once additional optimizations have been made, as genetic algorithms are way more demanding than training per backpropagation.
  arXiv  Detail & Related papers  (2022-10-20T18:41:57Z)
• Neural Networks Enhancement with Logical Knowledge [83.9217787335878]
  We propose an extension of KENN for relational data. The results show that KENN is capable of increasing the performances of the underlying neural network even in the presence relational data.
  arXiv  Detail & Related papers  (2020-09-13T21:12:20Z)
• Modeling from Features: a Mean-field Framework for Over-parameterized Deep Neural Networks [54.27962244835622]
  This paper proposes a new mean-field framework for over- parameterized deep neural networks (DNNs) In this framework, a DNN is represented by probability measures and functions over its features in the continuous limit. We illustrate the framework via the standard DNN and the Residual Network (Res-Net) architectures.
  arXiv  Detail & Related papers  (2020-07-03T01:37:16Z)
• An Ode to an ODE [78.97367880223254]
  We present a new paradigm for Neural ODE algorithms, called ODEtoODE, where time-dependent parameters of the main flow evolve according to a matrix flow on the group O(d) This nested system of two flows provides stability and effectiveness of training and provably solves the gradient vanishing-explosion problem.
  arXiv  Detail & Related papers  (2020-06-19T22:05:19Z)
• Binarizing MobileNet via Evolution-based Searching [66.94247681870125]
  We propose a use of evolutionary search to facilitate the construction and training scheme when binarizing MobileNet. Inspired by one-shot architecture search frameworks, we manipulate the idea of group convolution to design efficient 1-Bit Convolutional Neural Networks (CNNs) Our objective is to come up with a tiny yet efficient binary neural architecture by exploring the best candidates of the group convolution.
  arXiv  Detail & Related papers  (2020-05-13T13:25:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.