How Syntax Specialization Emerges in Language Models

• URL: http://arxiv.org/abs/2505.19548v1
• Date: Mon, 26 May 2025 06:11:18 GMT
• Title: How Syntax Specialization Emerges in Language Models
• Authors: Xufeng Duan, Zhaoqian Yao, Yunhao Zhang, Shaonan Wang, Zhenguang G. Cai,
• Abstract summary: Large language models (LLMs) have been found to develop surprising internal specializations.<n>Individual neurons, attention heads, and circuits selectively become sensitive to syntactic structure.<n>How this specialization emerges during training and what influences its development remains largely unknown.
• Score: 9.177796238194984
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Large language models (LLMs) have been found to develop surprising internal specializations: Individual neurons, attention heads, and circuits become selectively sensitive to syntactic structure, reflecting patterns observed in the human brain. While this specialization is well-documented, how it emerges during training and what influences its development remains largely unknown. In this work, we tap into the black box of specialization by tracking its formation over time. By quantifying internal syntactic consistency across minimal pairs from various syntactic phenomena, we identify a clear developmental trajectory: Syntactic sensitivity emerges gradually, concentrates in specific layers, and exhibits a 'critical period' of rapid internal specialization. This process is consistent across architectures and initialization parameters (e.g., random seeds), and is influenced by model scale and training data. We therefore reveal not only where syntax arises in LLMs but also how some models internalize it during training. To support future research, we will release the code, models, and training checkpoints upon acceptance.

Related papers

• A Brain-like Synergistic Core in LLMs Drives Behaviour and Learning [50.68188138112555]
  We show that large language models spontaneously develop synergistic cores.<n>We find that areas in middle layers exhibit synergistic processing while early and late layers rely on redundancy.<n>This convergence suggests that synergistic information processing is a fundamental property of intelligence.
  arXiv  Detail & Related papers  (2026-01-11T10:48:35Z)
• Distributed Specialization: Rare-Token Neurons in Large Language Models [8.13000021263958]
  Large language models (LLMs) struggle with representing and generating rare tokens despite their importance in specialized domains.<n>We investigate whether LLMs develop internal specialization mechanisms through discrete modular architectures or distributed parameter-level differentiation domains.
  arXiv  Detail & Related papers  (2025-09-25T13:49:38Z)
• The Other Mind: How Language Models Exhibit Human Temporal Cognition [9.509386631514122]
  Large Language Models (LLMs) exhibit certain cognitive patterns similar to those of humans that are not directly specified in training data.<n>We find that larger models spontaneously establish a subjective temporal reference point and adhere to the Weber-Fechner law.<n>Using pre-trained embedding models, we found that the training corpus itself possesses an inherent, non-linear temporal structure.
  arXiv  Detail & Related papers  (2025-07-21T17:59:01Z)
• The Emergence of Abstract Thought in Large Language Models Beyond Any Language [95.50197866832772]
  Large language models (LLMs) function effectively across a diverse range of languages.<n>Preliminary studies observe that the hidden activations of LLMs often resemble English, even when responding to non-English prompts.<n>Recent results show strong multilingual performance, even surpassing English performance on specific tasks in other languages.
  arXiv  Detail & Related papers  (2025-06-11T16:00:54Z)
• Cross-Lingual Generalization and Compression: From Language-Specific to Shared Neurons [20.13484267765109]
  We study how multilingual language models evolve during pre-training.<n>We observe a transition from uniform language identification capabilities across layers to more specialized layer functions.<n>We identify specific neurons that emerge as increasingly reliable predictors for the same concepts across languages.
  arXiv  Detail & Related papers  (2025-06-02T13:06:30Z)
• The Birth of Knowledge: Emergent Features across Time, Space, and Scale in Large Language Models [3.541570601342306]
  This paper studies the emergence of interpretable categorical features within large language models (LLMs)<n>Using sparse autoencoders for mechanistic interpretability, we identify when and where specific semantic concepts emerge within neural activations.
  arXiv  Detail & Related papers  (2025-05-26T02:59:54Z)
• Mechanistic Understanding and Mitigation of Language Confusion in English-Centric Large Language Models [49.09746599881631]
  We present the first mechanistic interpretability study of language confusion.<n>We show that confusion points (CPs) are central to this phenomenon.<n>We show that editing a small set of critical neurons, identified via comparative analysis with multilingual-tuned models, substantially mitigates confusion.
  arXiv  Detail & Related papers  (2025-05-22T11:29:17Z)
• Emergent Specialization: Rare Token Neurons in Language Models [5.946977198458224]
  Large language models struggle with representing and generating rare tokens despite their importance in specialized domains.<n>In this study, we identify neuron structures with exceptionally strong influence on language model's prediction of rare tokens, termed as rare token neurons.
  arXiv  Detail & Related papers  (2025-05-19T08:05:13Z)
• Analysis of Argument Structure Constructions in a Deep Recurrent Language Model [0.0]
  We explore the representation and processing of Argument Structure Constructions (ASCs) in a recurrent neural language model. Our results show that sentence representations form distinct clusters corresponding to the four ASCs across all hidden layers. This indicates that even a relatively simple, brain-constrained recurrent neural network can effectively differentiate between various construction types.
  arXiv  Detail & Related papers  (2024-08-06T09:27:41Z)
• Holmes: A Benchmark to Assess the Linguistic Competence of Language Models [59.627729608055006]
  We introduce Holmes, a new benchmark designed to assess language models (LMs) linguistic competence. We use computation-based probing to examine LMs' internal representations regarding distinct linguistic phenomena. As a result, we meet recent calls to disentangle LMs' linguistic competence from other cognitive abilities.
  arXiv  Detail & Related papers  (2024-04-29T17:58:36Z)
• Subspace Chronicles: How Linguistic Information Emerges, Shifts and Interacts during Language Model Training [56.74440457571821]
  We analyze tasks covering syntax, semantics and reasoning, across 2M pre-training steps and five seeds. We identify critical learning phases across tasks and time, during which subspaces emerge, share information, and later disentangle to specialize. Our findings have implications for model interpretability, multi-task learning, and learning from limited data.
  arXiv  Detail & Related papers  (2023-10-25T09:09:55Z)
• Exploring Memorization in Fine-tuned Language Models [53.52403444655213]
  We conduct the first comprehensive analysis to explore language models' memorization during fine-tuning across tasks. Our studies with open-sourced and our own fine-tuned LMs across various tasks indicate that memorization presents a strong disparity among different fine-tuning tasks. We provide an intuitive explanation of this task disparity via sparse coding theory and unveil a strong correlation between memorization and attention score distribution.
  arXiv  Detail & Related papers  (2023-10-10T15:41:26Z)
• Dynamic Spatio-Temporal Specialization Learning for Fine-Grained Action Recognition [19.562218963941227]
  We derive inspiration from the human visual system which contains specialized regions that are dedicated towards handling specific tasks. We design a novel Dynamic Dynamic Spatio-Temporal subset (DSTS) module, which consists of specialized neurons that are only activated for a subset of samples that are highly similar. We design an UpstreamDownstream Learning algorithm to optimize our model's dynamic decisions during training, improving the performance of our DSTS module.
  arXiv  Detail & Related papers  (2022-09-03T13:59:49Z)
• Mechanisms for Handling Nested Dependencies in Neural-Network Language Models and Humans [75.15855405318855]
  We studied whether a modern artificial neural network trained with "deep learning" methods mimics a central aspect of human sentence processing. Although the network was solely trained to predict the next word in a large corpus, analysis showed the emergence of specialized units that successfully handled local and long-distance syntactic agreement. We tested the model's predictions in a behavioral experiment where humans detected violations in number agreement in sentences with systematic variations in the singular/plural status of multiple nouns.
  arXiv  Detail & Related papers  (2020-06-19T12:00:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.