Understanding the Nature of Generative AI as Threshold Logic in High-Dimensional Space

• URL: http://arxiv.org/abs/2604.02476v1
• Date: Thu, 02 Apr 2026 19:20:21 GMT
• Title: Understanding the Nature of Generative AI as Threshold Logic in High-Dimensional Space
• Abstract summary: This paper examines the role of threshold logic in understanding generative artificial intelligence.<n>It argues that this shift has equally significant implications for understanding neural computation.
• Score: 0.7252027234425333
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper examines the role of threshold logic in understanding generative artificial intelligence. Threshold functions, originally studied in the 1960s in digital circuit synthesis, provide a structurally transparent model of neural computation: a weighted sum of inputs compared to a threshold, geometrically realized as a hyperplane partitioning a space. The paper shows that this operation undergoes a qualitative transition as dimensionality increases. In low dimensions, the perceptron acts as a determinate logical classifier, separating classes when possible, as decided by linear programming. In high dimensions, however, a single hyperplane can separate almost any configuration of points (Cover, 1965); the space becomes saturated with potential classifiers, and the perceptron shifts from a logical device to a navigational one, functioning as an indexical indicator in the sense of Peirce. The limitations of the perceptron identified by Minsky and Papert (1969) were historically addressed by introducing multilayer architectures. This paper considers an alternative path: increasing dimensionality while retaining a single threshold element. It argues that this shift has equally significant implications for understanding neural computation. The role of depth is reinterpreted as a mechanism for the sequential deformation of data manifolds through iterated threshold operations, preparing them for linear separability already afforded by high-dimensional geometry. The resulting triadic account - threshold function as ontological unit, dimensionality as enabling condition, and depth as preparatory mechanism - provides a unified perspective on generative AI grounded in established mathematics.

Related papers

• A Compositional Theory of Causally Masked Transformers [50.88598486616582]
  We develop a formalization that derives expressivity directly from the model's implemented dynamics.<n>Each attention head updates its own state independently within a layer.<n>A modified form of soft attention supports irreversible, checklist-like state.
  arXiv  Detail & Related papers  (2026-07-29T14:47:19Z)
• AI Engram: In Search of Memory Traces in Artificial Intelligence [19.383960470953145]
  This work introduces a geometric framework to identify such "AI engrams"<n>We derive a closed-form estimator that isolates individual memory traces from globally entangled parameters.<n>Results bridge theories of biological memory and artificial representation learning.
  arXiv  Detail & Related papers  (2026-06-12T22:36:52Z)
• Beyond LLMs, Sparse Distributed Memory, and Neuromorphics <A Hyper-Dimensional SRAM-CAM "VaCoAl" for Ultra-High Speed, Ultra-Low Power, and Low Cost> [18.56157183294801]
  We propose VaCoAl on ultra-high-dimensional binary spaces on ultra-high-dimensional RAM/DRAM-CAM.<n>It embeds a cognitive bound -- the Frontier Size -- into its architecture, ranking candidates by path-integral confidence.<n>We evaluate multi-hop reasoning on about 470k mentor-student relations from Wikidata, tracing up to 57 generations.
  arXiv  Detail & Related papers  (2026-04-13T16:13:17Z)
• Epistemology of Generative AI: The Geometry of Knowing [0.7252027234425333]
  Generative AI presents an unprecedented challenge to our understanding of knowledge and its production.<n>This paper argues that the missing account must begin with a paradigmatic break that has not yet received adequate philosophical attention.
  arXiv  Detail & Related papers  (2026-02-19T06:34:34Z)
• Low-Dimensional Execution Manifolds in Transformer Learning Dynamics: Evidence from Modular Arithmetic Tasks [0.0]
  We investigate the structure of learning dynamics in transformer models through carefully controlled arithmetic tasks.<n>Our results suggest a unifying geometric framework for understanding transformer learning.
  arXiv  Detail & Related papers  (2026-02-11T03:57:46Z)
• On the Emergence of Induction Heads for In-Context Learning [121.64612469118464]
  We study the emergence of induction heads, a previously identified mechanism in two-layer transformers.<n>We explain the origin of this structure using a minimal ICL task formulation and a modified transformer architecture.
  arXiv  Detail & Related papers  (2025-11-02T18:12:06Z)
• The Origins of Representation Manifolds in Large Language Models [52.68554895844062]
  We show that cosine similarity in representation space may encode the intrinsic geometry of a feature through shortest, on-manifold paths.<n>The critical assumptions and predictions of the theory are validated on text embeddings and token activations of large language models.
  arXiv  Detail & Related papers  (2025-05-23T13:31:22Z)
• Neural Networks as Universal Finite-State Machines: A Constructive Deterministic Finite Automaton Theory [0.0]
  We establish feedforward neural networks as universal finite-state machines (N-FSMs)<n>Our results prove that finite-depth ReLU and threshold networks can exactly simulate deterministic finite automata (DFAs)<n>We formalize the expressivity boundary: fixed-depth feedforward networks cannot recognize non-regular languages requiring memory.
  arXiv  Detail & Related papers  (2025-05-16T21:01:34Z)
• Average gradient outer product as a mechanism for deep neural collapse [26.939895223897572]
  Deep Neural Collapse (DNC) refers to the surprisingly rigid structure of the data representations in the final layers of Deep Neural Networks (DNNs)<n>In this work, we introduce a data-dependent setting where DNC forms due to feature learning through the average gradient outer product (AGOP)<n>We show that the right singular vectors and values of the weights can be responsible for the majority of within-class variability collapse for neural networks trained in the feature learning regime.
  arXiv  Detail & Related papers  (2024-02-21T11:40:27Z)
• LOGICSEG: Parsing Visual Semantics with Neural Logic Learning and Reasoning [73.98142349171552]
  LOGICSEG is a holistic visual semantic that integrates neural inductive learning and logic reasoning with both rich data and symbolic knowledge. During fuzzy logic-based continuous relaxation, logical formulae are grounded onto data and neural computational graphs, hence enabling logic-induced network training. These designs together make LOGICSEG a general and compact neural-logic machine that is readily integrated into existing segmentation models.
  arXiv  Detail & Related papers  (2023-09-24T05:43:19Z)
• High-dimensional separability for one- and few-shot learning [58.8599521537]
  This work is driven by a practical question, corrections of Artificial Intelligence (AI) errors. Special external devices, correctors, are developed. They should provide quick and non-iterative system fix without modification of a legacy AI system. New multi-correctors of AI systems are presented and illustrated with examples of predicting errors and learning new classes of objects by a deep convolutional neural network.
  arXiv  Detail & Related papers  (2021-06-28T14:58:14Z)
• EigenGAN: Layer-Wise Eigen-Learning for GANs [84.33920839885619]
  EigenGAN is able to unsupervisedly mine interpretable and controllable dimensions from different generator layers. By traversing the coefficient of a specific eigen-dimension, the generator can produce samples with continuous changes corresponding to a specific semantic attribute.
  arXiv  Detail & Related papers  (2021-04-26T11:14:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.