Neural Echos: Depthwise Convolutional Filters Replicate Biological Receptive Fields

• URL: http://arxiv.org/abs/2401.10178v1
• Date: Thu, 18 Jan 2024 18:06:22 GMT
• Title: Neural Echos: Depthwise Convolutional Filters Replicate Biological Receptive Fields
• Authors: Zahra Babaiee, Peyman M. Kiasari, Daniela Rus, Radu Grosu
• Abstract summary: We present evidence suggesting that depthwise convolutional kernels are effectively replicating the biological receptive fields observed in the mammalian retina. We propose a scheme that draws inspiration from the biological receptive fields.
• Score: 56.69755544814834
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: In this study, we present evidence suggesting that depthwise convolutional kernels are effectively replicating the structural intricacies of the biological receptive fields observed in the mammalian retina. We provide analytics of trained kernels from various state-of-the-art models substantiating this evidence. Inspired by this intriguing discovery, we propose an initialization scheme that draws inspiration from the biological receptive fields. Experimental analysis of the ImageNet dataset with multiple CNN architectures featuring depthwise convolutions reveals a marked enhancement in the accuracy of the learned model when initialized with biologically derived weights. This underlies the potential for biologically inspired computational models to further our understanding of vision processing systems and to improve the efficacy of convolutional networks.

Related papers

• Deep Continuous Networks [21.849285945717632]
  We propose deep continuous networks (DCNs), which combine spatially continuous filters, with the continuous depth framework of neural ODEs. This allows us to learn the spatial support of the filters during training, as well as model the continuous evolution of feature maps, linking DCNs closely to biological models. We show that DCNs are versatile and highly applicable to standard image classification and reconstruction problems, where they improve parameter and data efficiency, and allow for meta-parametrization.
  arXiv  Detail & Related papers  (2024-02-02T16:50:18Z)
• Unveiling the Unseen: Identifiable Clusters in Trained Depthwise Convolutional Kernels [56.69755544814834]
  Recent advances in depthwise-separable convolutional neural networks (DS-CNNs) have led to novel architectures. This paper reveals another striking property of DS-CNN architectures: discernible and explainable patterns emerge in their trained depthwise convolutional kernels in all layers.
  arXiv  Detail & Related papers  (2024-01-25T19:05:53Z)
• Morphological Profiling for Drug Discovery in the Era of Deep Learning [13.307277432389496]
  We provide a comprehensive overview of the recent advances in the field of morphological profiling. We place a particular emphasis on the application of deep learning in this pipeline.
  arXiv  Detail & Related papers  (2023-12-13T05:08:32Z)
• Tertiary Lymphoid Structures Generation through Graph-based Diffusion [54.37503714313661]
  In this work, we leverage state-of-the-art graph-based diffusion models to generate biologically meaningful cell-graphs. We show that the adopted graph diffusion model is able to accurately learn the distribution of cells in terms of their tertiary lymphoid structures (TLS) content.
  arXiv  Detail & Related papers  (2023-10-10T14:37:17Z)
• Contrastive-Signal-Dependent Plasticity: Self-Supervised Learning in Spiking Neural Circuits [61.94533459151743]
  This work addresses the challenge of designing neurobiologically-motivated schemes for adjusting the synapses of spiking networks. Our experimental simulations demonstrate a consistent advantage over other biologically-plausible approaches when training recurrent spiking networks.
  arXiv  Detail & Related papers  (2023-03-30T02:40:28Z)
• Multi-level Data Representation For Training Deep Helmholtz Machines [0.0]
  We guide the learning of a biologically plausible generative model called the Helmholtz Machine in complex search spaces using the Human Image Perception mechanism. We propose to overcome this problem, by providing the network's hidden layers with visual queues at different resolutions using a Multi-level Data representation. The results on several image datasets showed the model was able to not only obtain better overall quality but also a wider diversity in the generated images.
  arXiv  Detail & Related papers  (2022-10-26T16:55:40Z)
• Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
  We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs. By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
  arXiv  Detail & Related papers  (2022-02-01T17:11:13Z)
• Deep Unrolled Recovery in Sparse Biological Imaging [62.997667081978825]
  Deep algorithm unrolling is a model-based approach to develop deep architectures that combine the interpretability of iterative algorithms with the performance gains of supervised deep learning. This framework is well-suited to applications in biological imaging, where physics-based models exist to describe the measurement process and the information to be recovered is often highly structured.
  arXiv  Detail & Related papers  (2021-09-28T20:22:44Z)
• Evolution Is All You Need: Phylogenetic Augmentation for Contrastive Learning [1.7188280334580197]
  Self-supervised representation learning of biological sequence embeddings alleviates computational resource constraints on downstream tasks. We show that contrastive learning using evolutionary phylogenetic augmentation can be used as a representation learning objective.
  arXiv  Detail & Related papers  (2020-12-25T01:35:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.