The Process of Categorical Clipping at the Core of the Genesis of Concepts in Synthetic Neural Cognition

• URL: http://arxiv.org/abs/2502.15710v1
• Date: Tue, 21 Jan 2025 11:32:39 GMT
• Title: The Process of Categorical Clipping at the Core of the Genesis of Concepts in Synthetic Neural Cognition
• Authors: Michael Pichat, William Pogrund, Armanush Gasparian, Paloma Pichat, Samuel Demarchi, Michael Veillet-Guillem, Martin Corbet, Théo Dasilva,
• Abstract summary: This article investigates, within the field of neuropsychology of artificial intelligence, the process of categorical segmentation performed by language models.<n>This process involves, across different neural layers, the creation of new functional categorical dimensions to analyze the input textual data and perform the required tasks.<n>We explore several cognitive characteristics of this synthetic clipping in an exploratory manner.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: This article investigates, within the field of neuropsychology of artificial intelligence, the process of categorical segmentation performed by language models. This process involves, across different neural layers, the creation of new functional categorical dimensions to analyze the input textual data and perform the required tasks. Each neuron in a multilayer perceptron (MLP) network is associated with a specific category, generated by three factors carried by the neural aggregation function: categorical priming, categorical attention, and categorical phasing. At each new layer, these factors govern the formation of new categories derived from the categories of precursor neurons. Through a process of categorical clipping, these new categories are created by selectively extracting specific subdimensions from the preceding categories, constructing a distinction between a form and a categorical background. We explore several cognitive characteristics of this synthetic clipping in an exploratory manner: categorical reduction, categorical selectivity, separation of initial embedding dimensions, and segmentation of categorical zones.

Related papers

• Intra-neuronal attention within language models Relationships between activation and semantics [0.0]
  This study investigates the ability of perceptron-type neurons in language models to perform intra-neuronal attention. The objective of this work is to determine to what extent formal neurons can establish a homomorphic relationship between activation-based and categorical segmentations.
  arXiv  Detail & Related papers  (2025-03-17T09:47:11Z)
• Category-Selective Neurons in Deep Networks: Comparing Purely Visual and Visual-Language Models [23.309064032922507]
  Category-selective regions in the human brain play a crucial role in high-level visual processing.<n>We investigate whether artificial neural networks (ANNs) exhibit similar category-selective neurons.<n>Our study provides insights into how ANNs mirror biological vision and how multimodal learning influences category-selective representations.
  arXiv  Detail & Related papers  (2025-02-23T06:15:51Z)
• Neuropsychology and Explainability of AI: A Distributional Approach to the Relationship Between Activation Similarity of Neural Categories in Synthetic Cognition [0.11235145048383502]
  We propose an approach to explainability of artificial neural networks that involves using concepts from human cognitive tokens. We show that the categorical segment created by a neuron is actually the result of a superposition of categorical sub-dimensions within its input vector space.
  arXiv  Detail & Related papers  (2024-10-23T05:27:09Z)
• Information theoretic study of the neural geometry induced by category learning [0.0]
  We take an information theoretic approach to assess the efficiency of the representations induced by category learning. One main consequence is that category learning induces an expansion of neural space near decision boundaries.
  arXiv  Detail & Related papers  (2023-11-27T10:16:22Z)
• Multilayer Multiset Neuronal Networks -- MMNNs [55.2480439325792]
  The present work describes multilayer multiset neuronal networks incorporating two or more layers of coincidence similarity neurons. The work also explores the utilization of counter-prototype points, which are assigned to the image regions to be avoided.
  arXiv  Detail & Related papers  (2023-08-28T12:55:13Z)
• Understanding Imbalanced Semantic Segmentation Through Neural Collapse [81.89121711426951]
  We show that semantic segmentation naturally brings contextual correlation and imbalanced distribution among classes. We introduce a regularizer on feature centers to encourage the network to learn features closer to the appealing structure. Our method ranks 1st and sets a new record on the ScanNet200 test leaderboard.
  arXiv  Detail & Related papers  (2023-01-03T13:51:51Z)
• Automatically Discovering Novel Visual Categories with Self-supervised Prototype Learning [68.63910949916209]
  This paper tackles the problem of novel category discovery (NCD), which aims to discriminate unknown categories in large-scale image collections. We propose a novel adaptive prototype learning method consisting of two main stages: prototypical representation learning and prototypical self-training. We conduct extensive experiments on four benchmark datasets and demonstrate the effectiveness and robustness of the proposed method with state-of-the-art performance.
  arXiv  Detail & Related papers  (2022-08-01T16:34:33Z)
• Domain Adaptive Nuclei Instance Segmentation and Classification via Category-aware Feature Alignment and Pseudo-labelling [65.40672505658213]
  We propose a novel deep neural network, namely Category-Aware feature alignment and Pseudo-Labelling Network (CAPL-Net) for UDA nuclei instance segmentation and classification. Our approach outperforms state-of-the-art UDA methods with a remarkable margin.
  arXiv  Detail & Related papers  (2022-07-04T07:05:06Z)
• Self-Supervised Graph Representation Learning for Neuronal Morphologies [75.38832711445421]
  We present GraphDINO, a data-driven approach to learn low-dimensional representations of 3D neuronal morphologies from unlabeled datasets. We show, in two different species and across multiple brain areas, that this method yields morphological cell type clusterings on par with manual feature-based classification by experts. Our method could potentially enable data-driven discovery of novel morphological features and cell types in large-scale datasets.
  arXiv  Detail & Related papers  (2021-12-23T12:17:47Z)
• Modeling Category-Selective Cortical Regions with Topographic Variational Autoencoders [72.15087604017441]
  Category-selectivity describes the observation that certain spatially localized areas of the cerebral cortex tend to respond robustly and selectively to stimuli from specific limited categories. We leverage the newly introduced Topographic Variational Autoencoder to model of the emergence of such localized category-selectivity in an unsupervised manner. We show preliminary results suggesting that our model yields a nested spatial hierarchy of increasingly abstract categories, analogous to observations from the human ventral temporal cortex.
  arXiv  Detail & Related papers  (2021-10-25T11:37:41Z)
• Enhancing efficiency of object recognition in different categorization levels by reinforcement learning in modular spiking neural networks [1.392250707100996]
  We propose a computational model for object recognition in different categorization levels. A spiking neural network equipped with the reinforcement learning rule is used as a module at each categorization level. According to the required information at each categorization level, the relevant band-pass filtered images are utilized.
  arXiv  Detail & Related papers  (2021-02-10T12:33:20Z)
• Categorical Perception: A Groundwork for Deep Learning [0.0]
  We study categorical effects in artificial neural networks. We show on both shallow and deep neural networks that category learning automatically induces categorical perception. An important outcome of our analysis is to provide a coherent and unifying view of the efficacy of the dropout regularization technique.
  arXiv  Detail & Related papers  (2020-12-10T09:41:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.