Factors Influencing the Emergence of Dependency Length Minimization in Neural Agent Simulations

• URL: http://arxiv.org/abs/2609.06025v1
• Date: Sat, 05 Sep 2026 11:03:54 GMT
• Title: Factors Influencing the Emergence of Dependency Length Minimization in Neural Agent Simulations
• Abstract summary: We study the emergence of DLM in artificial languages using a proposed language learning and communication framework based on recurrent neural networks (RNNs)<n>Our results reveal a complex interplay among these factors in shaping word order preferences in neural agents.<n>These findings suggest that limitations in human cognitive processing may indeed play a role in shaping DLM.
• Score: 9.253309436742928
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Given various grammatical options, language users prefer the word order choice that reduces the overall length of syntactic dependencies, a principle known as dependency length minimization (DLM). The origins of this preference remain an open question, particularly whether it originates from constraints on efficient information processing. Computational simulations provide a powerful approach to identifying the factors influencing the emergence of linguistic phenomena. However, previous simulations of DLM have not examined realistic interaction contexts and have produced mixed results. The present study investigates the emergence of DLM in artificial languages using a recently proposed language learning and communication framework based on recurrent neural networks (RNNs). In this framework, agents are trained to speak and interpret artificial languages and then use these languages to communicate. Using this framework, we study the impact of several factors related to processing limitations in a communicative setting, such as noise during listening, limited speaker capacity, and incremental sentence processing. Our results reveal a complex interplay among these factors in shaping word order preferences in neural agents. Specifically, in the full meaning space, agents regularize toward a single dominant word order, while in the half meaning space they show a short-before-long preference that only aligns with DLM in verb-initial languages. A consistent DLM preference emerges only when agents are subject to incremental processing pressure. These findings suggest that limitations in human cognitive processing may indeed play a role in shaping DLM. Our findings provide insights into the conditions under which neural models replicate human-like preferences and highlight the challenges of designing emergent communication models that capture human cognitive biases in language processing.

Related papers

• Interpreting Brain Responses to Language with Sparse Features from Language Models [5.261667457257713]
  We develop an encoding framework that replaces dense LM hidden states with hierarchically-organized autoencoder features.<n>We show that brain responses during language processing are not merely predictable from an arbitrary set of LM features.
  arXiv  Detail & Related papers  (2026-06-05T03:01:28Z)
• Unveiling the Influence of Amplifying Language-Specific Neurons [11.19692440351977]
  Language-specific neurons that strongly correlate with individual languages have been shown to influence model behavior by deactivating them.<n>This work investigates the effect of amplifying language-specific neurons through interventions across 18 languages.
  arXiv  Detail & Related papers  (2025-07-30T11:23:30Z)
• The Emergence of Abstract Thought in Large Language Models Beyond Any Language [95.50197866832772]
  Large language models (LLMs) function effectively across a diverse range of languages.<n>Preliminary studies observe that the hidden activations of LLMs often resemble English, even when responding to non-English prompts.<n>Recent results show strong multilingual performance, even surpassing English performance on specific tasks in other languages.
  arXiv  Detail & Related papers  (2025-06-11T16:00:54Z)
• Mechanistic Understanding and Mitigation of Language Confusion in English-Centric Large Language Models [56.61984030508691]
  We present the first mechanistic interpretability study of language confusion.<n>We show that confusion points (CPs) are central to this phenomenon.<n>We show that editing a small set of critical neurons, identified via comparative analysis with a multilingual-tuned counterpart, substantially mitigates confusion.
  arXiv  Detail & Related papers  (2025-05-22T11:29:17Z)
• When Less Language is More: Language-Reasoning Disentanglement Makes LLMs Better Multilingual Reasoners [111.50503126693444]
  We show that language-specific ablation consistently boosts multilingual reasoning performance.<n>Compared to post-training, our training-free ablation achieves comparable or superior results with minimal computational overhead.
  arXiv  Detail & Related papers  (2025-05-21T08:35:05Z)
• Simulating the Emergence of Differential Case Marking with Communicating Neural-Network Agents [2.184775414778289]
  Differential Case Marking (DCM) refers to the phenomenon where grammatical case marking is applied selectively based on semantic, pragmatic, or other factors.<n>In this study, we employ such a framework in which agents first acquire an artificial language before engaging in communicative interactions.<n>Using a very generic communication optimization algorithm and neural-network learners that have no prior experience with language or semantic preferences, our results demonstrate that learning alone does not lead to DCM.
  arXiv  Detail & Related papers  (2025-02-06T13:00:53Z)
• Developmental Predictive Coding Model for Early Infancy Mono and Bilingual Vocal Continual Learning [69.8008228833895]
  We propose a small-sized generative neural network equipped with a continual learning mechanism.<n>Our model prioritizes interpretability and demonstrates the advantages of online learning.
  arXiv  Detail & Related papers  (2024-12-23T10:23:47Z)
• How do Large Language Models Handle Multilingualism? [81.15060972112563]
  This study explores how large language models (LLMs) handle multilingualism. LLMs initially understand the query, converting multilingual inputs into English for task-solving. In the intermediate layers, they employ English for thinking and incorporate multilingual knowledge with self-attention and feed-forward structures.
  arXiv  Detail & Related papers  (2024-02-29T02:55:26Z)
• Explaining Interactions Between Text Spans [50.70253702800355]
  Reasoning over spans of tokens from different parts of the input is essential for natural language understanding. We introduce SpanEx, a dataset of human span interaction explanations for two NLU tasks: NLI and FC. We then investigate the decision-making processes of multiple fine-tuned large language models in terms of the employed connections between spans.
  arXiv  Detail & Related papers  (2023-10-20T13:52:37Z)
• Let Models Speak Ciphers: Multiagent Debate through Embeddings [84.20336971784495]
  We introduce CIPHER (Communicative Inter-Model Protocol Through Embedding Representation) to address this issue. By deviating from natural language, CIPHER offers an advantage of encoding a broader spectrum of information without any modification to the model weights. This showcases the superiority and robustness of embeddings as an alternative "language" for communication among LLMs.
  arXiv  Detail & Related papers  (2023-10-10T03:06:38Z)
• Demystifying Neural Language Models' Insensitivity to Word-Order [7.72780997900827]
  We investigate the insensitivity of natural language models to word-order by quantifying perturbations. We find that neural language models require local ordering more so than the global ordering of tokens.
  arXiv  Detail & Related papers  (2021-07-29T13:34:20Z)
• Influence Paths for Characterizing Subject-Verb Number Agreement in LSTM Language Models [22.826154706036995]
  LSTM-based recurrent neural networks are the state-of-the-art for many natural language processing (NLP) tasks. Lacking this understanding, the generality of LSTM performance on this task and their suitability for related tasks remains uncertain. We introduce *influence paths*, a causal account of structural properties as carried by paths across gates and neurons of a recurrent neural network.
  arXiv  Detail & Related papers  (2020-05-03T21:10:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.