Efficiency Bottlenecks of Convolutional Kolmogorov-Arnold Networks: A Comprehensive Scrutiny with ImageNet, AlexNet, LeNet and Tabular Classification

• URL: http://arxiv.org/abs/2501.15757v2
• Date: Tue, 28 Jan 2025 04:26:12 GMT
• Title: Efficiency Bottlenecks of Convolutional Kolmogorov-Arnold Networks: A Comprehensive Scrutiny with ImageNet, AlexNet, LeNet and Tabular Classification
• Authors: Ashim Dahal, Saydul Akbar Murad, Nick Rahimi,
• Abstract summary: We train Convolutional Kolmogorov Arnold Networks (CKANs) with the ImageNet-1k dataset with 1.3 million images.<n>We show that the CKANs perform fair yet slower than CNNs in small size dataset like MoA and MNIST but are not nearly comparable as the dataset gets larger and more complex like the ImageNet.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Algorithmic level developments like Convolutional Neural Networks, transformers, attention mechanism, Retrieval Augmented Generation and so on have changed Artificial Intelligence. Recent such development was observed by Kolmogorov-Arnold Networks that suggested to challenge the fundamental concept of a Neural Network, thus change Multilayer Perceptron, and Convolutional Neural Networks. They received a good reception in terms of scientific modeling, yet had some drawbacks in terms of efficiency. In this paper, we train Convolutional Kolmogorov Arnold Networks (CKANs) with the ImageNet-1k dataset with 1.3 million images, MNIST dataset with 60k images and a tabular biological science related MoA dataset and test the promise of CKANs in terms of FLOPS, Inference Time, number of trainable parameters and training time against the accuracy, precision, recall and f-1 score they produce against the standard industry practice on CNN models. We show that the CKANs perform fair yet slower than CNNs in small size dataset like MoA and MNIST but are not nearly comparable as the dataset gets larger and more complex like the ImageNet. The code implementation of this paper can be found on the link: \href{https://github.com/ashimdahal/Study-of-Convolutional-Kolmogorov-Arnold-networks}{https://github.com/ashimdahal/Study-of-Convolutional-Kolmogorov-Arnold-networks}

Related papers

• Kolmogorov-Arnold Network Autoencoders [0.0]
  Kolmogorov-Arnold Networks (KANs) are promising alternatives to Multi-Layer Perceptrons (MLPs) KANs align closely with the Kolmogorov-Arnold representation theorem, potentially enhancing both model accuracy and interpretability. Our results demonstrate that KAN-based autoencoders achieve competitive performance in terms of reconstruction accuracy.
  arXiv  Detail & Related papers  (2024-10-02T22:56:00Z)
• Kolmogorov-Arnold Convolutions: Design Principles and Empirical Studies [0.0]
  This paper explores the application of Kolmogorov-Arnold Networks (KANs) in the domain of computer vision (CV) We propose a parameter-efficient design for Kolmogorov-Arnold convolutional layers and a parameter-efficient finetuning algorithm for pre-trained KAN models. We provide empirical evaluations conducted on MNIST, CIFAR10, CIFAR100, Tiny ImageNet, ImageNet1k, and HAM10000 datasets for image classification tasks.
  arXiv  Detail & Related papers  (2024-07-01T08:49:33Z)
• Enhancing Small Object Encoding in Deep Neural Networks: Introducing Fast&Focused-Net with Volume-wise Dot Product Layer [0.0]
  We introduce Fast&Focused-Net, a novel deep neural network architecture tailored for encoding small objects into fixed-length feature vectors. Fast&Focused-Net employs a series of our newly proposed layer, the Volume-wise Dot Product (VDP) layer, designed to address several inherent limitations of CNNs. For small object classification tasks, our network outperformed state-of-the-art methods on datasets such as CIFAR-10, CIFAR-100, STL-10, SVHN-Cropped, and Fashion-MNIST. In the context of larger image classification, when combined with a transformer encoder (ViT
  arXiv  Detail & Related papers  (2024-01-18T09:31:25Z)
• Convolutional Neural Networks Exploiting Attributes of Biological Neurons [7.3517426088986815]
  Deep neural networks like Convolutional Neural Networks (CNNs) have emerged as front-runners, often surpassing human capabilities. Here, we integrate the principles of biological neurons in certain layer(s) of CNNs. We aim to extract image features to use as input to CNNs, hoping to enhance training efficiency and achieve better accuracy.
  arXiv  Detail & Related papers  (2023-11-14T16:58:18Z)
• RIC-CNN: Rotation-Invariant Coordinate Convolutional Neural Network [56.42518353373004]
  We propose a new convolutional operation, called Rotation-Invariant Coordinate Convolution (RIC-C) By replacing all standard convolutional layers in a CNN with the corresponding RIC-C, a RIC-CNN can be derived. It can be observed that RIC-CNN achieves the state-of-the-art classification on the rotated test dataset of MNIST.
  arXiv  Detail & Related papers  (2022-11-21T19:27:02Z)
• InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions [95.94629864981091]
  This work presents a new large-scale CNN-based foundation model, termed InternImage, which can obtain the gain from increasing parameters and training data like ViTs. The proposed InternImage reduces the strict inductive bias of traditional CNNs and makes it possible to learn stronger and more robust patterns with large-scale parameters from massive data like ViTs.
  arXiv  Detail & Related papers  (2022-11-10T18:59:04Z)
• Neural Attentive Circuits [93.95502541529115]
  We introduce a general purpose, yet modular neural architecture called Neural Attentive Circuits (NACs) NACs learn the parameterization and a sparse connectivity of neural modules without using domain knowledge. NACs achieve an 8x speedup at inference time while losing less than 3% performance.
  arXiv  Detail & Related papers  (2022-10-14T18:00:07Z)
• CompNet: A Designated Model to Handle Combinations of Images and Designed features [0.24596929878045565]
  We propose a new structure of CNN-based model: CompNet, a composite convolutional neural network. With the use of this structure on classification tasks, the results indicate that our approach has the capability to significantly reduce overfitting.
  arXiv  Detail & Related papers  (2022-09-28T22:43:22Z)
• Portuguese Man-of-War Image Classification with Convolutional Neural Networks [58.720142291102135]
  Portuguese man-of-war (PMW) is a gelatinous organism with long tentacles capable of causing severe burns. This paper reports on the use of convolutional neural networks for recognizing PMW images from the Instagram social media.
  arXiv  Detail & Related papers  (2022-07-04T03:06:45Z)
• Learning Convolutional Neural Networks in the Frequency Domain [33.902889724984746]
  We propose a novel neural network model, namely CEMNet, that can be trained in frequency domain. We introduce Weight Fixation Mechanism to alleviate over-fitting, and analyze the working behavior of Batch Normalization, Leaky ReLU and Dropout. Experimental results imply that CEMNet works well in frequency domain, and achieve good performance on MNIST and CIFAR-10 databases.
  arXiv  Detail & Related papers  (2022-04-14T03:08:40Z)
• Lost Vibration Test Data Recovery Using Convolutional Neural Network: A Case Study [0.0]
  This paper proposes a CNN algorithm for the Alamosa Canyon Bridge as a real structure. Three different CNN models were considered to predict one and two malfunctioned sensors. The accuracy of the model was increased by adding a convolutional layer.
  arXiv  Detail & Related papers  (2022-04-11T23:24:03Z)
• Binarizing MobileNet via Evolution-based Searching [66.94247681870125]
  We propose a use of evolutionary search to facilitate the construction and training scheme when binarizing MobileNet. Inspired by one-shot architecture search frameworks, we manipulate the idea of group convolution to design efficient 1-Bit Convolutional Neural Networks (CNNs) Our objective is to come up with a tiny yet efficient binary neural architecture by exploring the best candidates of the group convolution.
  arXiv  Detail & Related papers  (2020-05-13T13:25:51Z)
• DRU-net: An Efficient Deep Convolutional Neural Network for Medical Image Segmentation [2.3574651879602215]
  Residual network (ResNet) and densely connected network (DenseNet) have significantly improved the training efficiency and performance of deep convolutional neural networks (DCNNs) We propose an efficient network architecture by considering advantages of both networks.
  arXiv  Detail & Related papers  (2020-04-28T12:16:24Z)
• Improved Residual Networks for Image and Video Recognition [98.10703825716142]
  Residual networks (ResNets) represent a powerful type of convolutional neural network (CNN) architecture. We show consistent improvements in accuracy and learning convergence over the baseline. Our proposed approach allows us to train extremely deep networks, while the baseline shows severe optimization issues.
  arXiv  Detail & Related papers  (2020-04-10T11:09:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.