Lost in Reordering: Structural Sensitivity of Multilingual LLMs under Semantics-Preserving Perturbations

• URL: http://arxiv.org/abs/2609.03511v1
• Date: Thu, 03 Sep 2026 08:10:43 GMT
• Title: Lost in Reordering: Structural Sensitivity of Multilingual LLMs under Semantics-Preserving Perturbations
• Abstract summary: Large Language Models (LLMs) demonstrate strong multilingual reasoning performance, yet their robustness to semantics-preserving structural variation remains underexplored.<n>We investigate the structural sensitivity of multilingual LLMs using two linguistically grounded perturbation settings in Hindi and Malayalam.<n>We observe consistent and significant degradation in mathematical reasoning performance under structurally perturbed inputs.
• Score: 19.00420862366793
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large Language Models (LLMs) demonstrate strong multilingual reasoning performance, yet their robustness to semantics-preserving structural variation remains underexplored, particularly for relatively free word-order languages. We investigate the structural sensitivity of multilingual LLMs using two linguistically grounded perturbation settings in Hindi and Malayalam: constrained constituent reordering and active-passive voice transformation. We introduce a benchmark dataset IndicReStruct, with two variants, GSM8K-Reordered and GSM8K-Voice, constructed from GSM8K while preserving semantic meaning. Across six state-of-the-art LLMs and multiple prompting strategies, we observe consistent and significant degradation in mathematical reasoning performance under structurally perturbed inputs. To further understand these failures, we perform qualitative error analysis and mechanistic interpretability experiments using residual-stream activation patching. Our analyses show that reasoning failures frequently arise from disruptions in entity-quantity alignment and that intermediate transformer layers contribute most strongly toward reasoning restoration. Overall, our findings suggest that current multilingual LLMs remain highly sensitive to surface syntactic realization and lack robust compositional invariance under structurally different but semantically equivalent inputs.

Related papers

• Forbid Your Attention: Fooling Multimodal Large Language Models by Selectively Removing Intrinsic Focus in Spectral Domain [94.64605746540472]
  Multimodal large language models (MLLMs) have extended the capability of large language models (LLMs) to process more contextual multimodal information.<n>Recent studies reveal that MLLMs remain highly vulnerable to adversarial inputs, especially those targeting visual components.<n>We propose a novel phase-aware adversarial attack framework that explicitly restricts adversarial perturbations to structure-relevant phase regions.
  arXiv  Detail & Related papers  (2026-09-01T06:34:32Z)
• MultiHaluDet: Multilingual Hallucination Detection via LLM Hidden State Probing [1.3700362496838856]
  Hallucinations in Large Language Models (LLMs) represent a critical barrier to their reliable deployment.<n>We introduce MultiHaluDet, a novel three-stage stacking framework that detects multilingual hallucinations.<n>Our framework achieves state-of-the-art detection performance, reaching up to 98.55% AUROC on the English HaluEval and TriviaQA benchmarks.
  arXiv  Detail & Related papers  (2026-05-24T07:50:03Z)
• Compensating Visual Insufficiency with Stratified Language Guidance for Long-Tail Class Incremental Learning [50.795452298625996]
  Long-tail class incremental learning (LT CIL) remains highly challenging because the scarcity of samples in tail classes hampers their learning.<n>We exploit the informativeness and scalability of language knowledge to tackle these issues.<n>We introduce stratified adaptive language guidance, which leverages learnable weights to merge multi-scale semantic representations.
  arXiv  Detail & Related papers  (2026-03-23T08:50:00Z)
• Same Meaning, Different Scores: Lexical and Syntactic Sensitivity in LLM Evaluation [40.210132040677]
  This paper examines how controlled, truth-conditionally equivalent lexical and syntactic perturbations affect the absolute performance and relative ranking of 23 contemporary Large Language Models (LLMs)<n>Results show that lexical perturbations consistently induce substantial, statistically significant performance degradation across nearly all models and tasks, while syntactic perturbations have more heterogeneous effects, occasionally improving results.
  arXiv  Detail & Related papers  (2026-02-19T12:24:42Z)
• Unlocking Reasoning Capability on Machine Translation in Large Language Models [57.60641851466707]
  Reasoning-oriented large language models (RLMs) achieve strong gains on tasks such as mathematics and coding by generating explicit intermediate reasoning.<n>We systematically evaluate several open- and closed-weights RLMs on the WMT24++ benchmark.<n>We find that enabling explicit reasoning consistently degrades translation quality across languages and models.
  arXiv  Detail & Related papers  (2026-02-16T14:05:59Z)
• Stable Language Guidance for Vision-Language-Action Models [62.80963701282789]
  Residual Semantic Steering is a probabilistic framework that disentangles physical affordance from semantic execution.<n> RSS achieves state-of-the-art robustness, maintaining performance even under adversarial linguistic perturbations.
  arXiv  Detail & Related papers  (2026-01-07T16:16:10Z)
• Sensitivity of Small Language Models to Fine-tuning Data Contamination [0.0]
  Small Language Models (SLMs) are increasingly being deployed in resource-constrained environments.<n>We measure susceptibility to syntactic and semantic transformation types during instruction tuning.<n>Character reversal produces near-complete failure across all models regardless of size or family.<n>Semantic transformations demonstrate distinct threshold behaviors and greater resilience in core linguistic capabilities.
  arXiv  Detail & Related papers  (2025-11-10T06:44:29Z)
• Mechanistic Understanding and Mitigation of Language Confusion in English-Centric Large Language Models [56.61984030508691]
  We present the first mechanistic interpretability study of language confusion.<n>We show that confusion points (CPs) are central to this phenomenon.<n>We show that editing a small set of critical neurons, identified via comparative analysis with a multilingual-tuned counterpart, substantially mitigates confusion.
  arXiv  Detail & Related papers  (2025-05-22T11:29:17Z)
• LLM-based Translation Inference with Iterative Bilingual Understanding [52.46978502902928]
  We propose a novel Iterative Bilingual Understanding Translation method based on the cross-lingual capabilities of large language models (LLMs)<n>The cross-lingual capability of LLMs enables the generation of contextual understanding for both the source and target languages separately.<n>The proposed IBUT outperforms several strong comparison methods.
  arXiv  Detail & Related papers  (2024-10-16T13:21:46Z)
• Emergent Linguistic Structures in Neural Networks are Fragile [20.692540987792732]
  Large Language Models (LLMs) have been reported to have strong performance on natural language processing tasks. We propose a framework to assess the consistency and robustness of linguistic representations.
  arXiv  Detail & Related papers  (2022-10-31T15:43:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.