On the Emergence of Syntax by Means of Local Interaction

• URL: http://arxiv.org/abs/2604.17857v2
• Date: Tue, 21 Apr 2026 13:11:34 GMT
• Title: On the Emergence of Syntax by Means of Local Interaction
• Abstract summary: We present a concrete instance of syntactic processing on a minimal system.<n>The system is trained on the membership problem of an arithmetic-expression grammar.<n>After training, its internal $L times L$ grid spontaneously self-organizes into an ordered, spatially extended representation.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Can syntactic processing emerge spontaneously from purely local interaction? We present a concrete instance on a minimal system: an 18,658-parameter two-dimensional neural cellular automaton (NCA), supervised by nothing more than a 1-bit boundary signal, is trained on the membership problem of an arithmetic-expression grammar. After training, its internal $L \times L$ grid spontaneously self-organizes into an ordered, spatially extended representation that we name Proto-CKY. This representation satisfies three operational criteria for syntactic processing: expressive power beyond the regular languages, structural generalization beyond the training distribution, and an internal organization quantitatively aligned with grammatical structure (Pearson $r \approx 0.71$). It emerges independently on four context-free grammars and regenerates spontaneously after perturbation. Proto-CKY is functionally aligned with the CKY algorithm but formally distinct from it: it is a physical prototype, a concrete instantiation of a mathematical ideal on a physical substrate, and the systematic distance between the two carries information about the substrate itself.

Related papers

• Formal Properties of Language as Constraints on Neural Dynamics [0.20305676256390937]
  Current research annotates stimuli with linguistic variables and tests which saturate electrodes, voxels, or language-model layers predict neural activity.<n>Here, we show that algebraic properties of language specify invariants that mechanisms must preserve.<n>We propose a novel neural binding operation we term 'Meld': two constituent populations converge through shared synapses, integrate sublinearly, and Merge.
  arXiv  Detail & Related papers  (2026-09-13T08:41:56Z)
• Weave of Formal Thought [51.56484100374058]
  We introduce Weave of Formal Thought (WoFT), a paradigm uniting rigorous syntactic validation with learned structural representations.<n>We present a latent-variable fine-tuning method training the language model to interleave non-terminal grammar symbols directly into generation.<n>For Python, fine-tuning StarCoder2-3B with our RWS objective reduces per-token cross-entropy by 14.3% relative to a text-only SFT baseline.
  arXiv  Detail & Related papers  (2026-06-24T15:58:11Z)
• $\mathcal{S}^2$IT: Stepwise Syntax Integration Tuning for Large Language Models in Aspect Sentiment Quad Prediction [51.165432266846096]
  syntactic structure information is underutilized in the generative paradigm of large language models (LLMs)<n>S2IT progressively integrates syntactic structure knowledge into LLMs through a multi-step tuning process.<n> Experiments demonstrate that S2IT significantly improves state-of-the-art performance across multiple datasets.
  arXiv  Detail & Related papers  (2026-04-25T13:24:29Z)
• CAPSUL: A Comprehensive Human Protein Benchmark for Subcellular Localization [50.559485139565076]
  We introduce a new benchmark called $mathbfCAPSUL$, a dataset that integrates diverse 3D structural representations with fine-grained subcellular localization annotations.<n>We evaluate this benchmark using a variety of state-of-the-art sequence-based and structure-based models, showcasing the importance of involving structural features in this task.<n> Lastly, we showcase the powerful interpretability of structure-based methods through a case study on the Golgi apparatus.
  arXiv  Detail & Related papers  (2026-03-19T07:33:04Z)
• Separable neural architectures as a primitive for unified predictive and generative intelligence [0.7318787399845128]
  We formalise a representational class that unifies additive, quadratic and tensor-decomposed neural models.<n>SNAs impose a structural inductive bias that factorises high-dimensional mappings into low-composed components.<n>These results establish the separable neural architecture as the domain-agnostic primitive for predictive and generative intelligence.
  arXiv  Detail & Related papers  (2026-03-12T17:56:54Z)
• From Static Structures to Ensembles: Studying and Harnessing Protein Structure Tokenization [15.864659611818661]
  Protein structure tokenization converts 3D structures into discrete or vectorized representations.<n>Despite many recent works on structure tokenization, the properties of the underlying discrete representations are not well understood.<n>We show that the successful utilization of structural tokens in a language model for structure prediction depends on using rich, pre-trained sequence embeddings.
  arXiv  Detail & Related papers  (2025-11-13T07:58:24Z)
• Spatial Semantic Recurrent Mining for Referring Image Segmentation [63.34997546393106]
  We propose Stextsuperscript2RM to achieve high-quality cross-modality fusion. It follows a working strategy of trilogy: distributing language feature, spatial semantic recurrent coparsing, and parsed-semantic balancing. Our proposed method performs favorably against other state-of-the-art algorithms.
  arXiv  Detail & Related papers  (2024-05-15T00:17:48Z)
• Semantic Loss Functions for Neuro-Symbolic Structured Prediction [74.18322585177832]
  We discuss the semantic loss, which injects knowledge about such structure, defined symbolically, into training. It is agnostic to the arrangement of the symbols, and depends only on the semantics expressed thereby. It can be combined with both discriminative and generative neural models.
  arXiv  Detail & Related papers  (2024-05-12T22:18:25Z)
• Operational Quantum Mereology and Minimal Scrambling [3.499870393443268]
  We will attempt to answer what are the natural quantum subsystems which emerge out of a system's dynamical laws. We first define generalized tensor product structures (gTPS) in terms of observables, as dual pairs of an operator subalgebra $cal A$ and its commutant. We propose an operational criterion of minimal information scrambling at short time scales to dynamically select gTPS.
  arXiv  Detail & Related papers  (2022-12-29T15:14:38Z)
• System theoretic approach of information processing in nested cellular automata [0.0]
  The concept of information processing in regular structures based on multi-level processing in nested cellular automata is presented. In the model appear expressions similar to expressions used in the Special Relativity Theory.
  arXiv  Detail & Related papers  (2022-10-12T09:45:45Z)
• X-volution: On the unification of convolution and self-attention [52.80459687846842]
  We propose a multi-branch elementary module composed of both convolution and self-attention operation. The proposed X-volution achieves highly competitive visual understanding improvements.
  arXiv  Detail & Related papers  (2021-06-04T04:32:02Z)
• Syntactic representation learning for neural network based TTS with syntactic parse tree traversal [49.05471750563229]
  We propose a syntactic representation learning method based on syntactic parse tree to automatically utilize the syntactic structure information. Experimental results demonstrate the effectiveness of our proposed approach. For sentences with multiple syntactic parse trees, prosodic differences can be clearly perceived from the synthesized speeches.
  arXiv  Detail & Related papers  (2020-12-13T05:52:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.