Universal Conceptual Structure in Neural Translation: Probing NLLB-200's Multilingual Geometry

• URL: http://arxiv.org/abs/2603.02258v1
• Date: Fri, 27 Feb 2026 22:51:01 GMT
• Title: Universal Conceptual Structure in Neural Translation: Probing NLLB-200's Multilingual Geometry
• Authors: Kyle Elliott Mathewson,
• Abstract summary: We investigate the representation geometry of Meta's NLLB-200, a 200-language encoder-decoder Transformer.<n>We find that the model's embedding distances significantly correlate with phylogenetic distances from the Automated Similarity Judgment Program.<n>We release InterpretCognates, an open-source interactive toolkit for exploring these phenomena.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Do neural machine translation models learn language-universal conceptual representations, or do they merely cluster languages by surface similarity? We investigate this question by probing the representation geometry of Meta's NLLB-200, a 200-language encoder-decoder Transformer, through six experiments that bridge NLP interpretability with cognitive science theories of multilingual lexical organization. Using the Swadesh core vocabulary list embedded across 135 languages, we find that the model's embedding distances significantly correlate with phylogenetic distances from the Automated Similarity Judgment Program ($ρ= 0.13$, $p = 0.020$), demonstrating that NLLB-200 has implicitly learned the genealogical structure of human languages. We show that frequently colexified concept pairs from the CLICS database exhibit significantly higher embedding similarity than non-colexified pairs ($U = 42656$, $p = 1.33 \times 10^{-11}$, $d = 0.96$), indicating that the model has internalized universal conceptual associations. Per-language mean-centering of embeddings improves the between-concept to within-concept distance ratio by a factor of 1.19, providing geometric evidence for a language-neutral conceptual store analogous to the anterior temporal lobe hub identified in bilingual neuroimaging. Semantic offset vectors between fundamental concept pairs (e.g., man to woman, big to small) show high cross-lingual consistency (mean cosine = 0.84), suggesting that second-order relational structure is preserved across typologically diverse languages. We release InterpretCognates, an open-source interactive toolkit for exploring these phenomena, alongside a fully reproducible analysis pipeline.

Related papers

• Subword-Based Comparative Linguistics across 242 Languages Using Wikipedia Glottosets [0.1682277069379282]
  We present a large-scale comparative study of 242 Latin and Cyrillic-script languages using subword-based methodologies.<n>Our approach utilizes Wikipedia rank-based subword vectors to analyze vocabulary, lexical divergence, and language similarity at scale.
  arXiv  Detail & Related papers  (2026-01-26T18:55:28Z)
• Modality Matching Matters: Calibrating Language Distances for Cross-Lingual Transfer in URIEL+ [4.262015666551064]
  We introduce a framework for type-matched language distances.<n>We propose novel, structure-aware representations for each distance type.<n>We unify these signals into a robust, task-agnostic composite distance.
  arXiv  Detail & Related papers  (2025-10-22T03:59:19Z)
• Mechanistic Understanding and Mitigation of Language Confusion in English-Centric Large Language Models [56.61984030508691]
  We present the first mechanistic interpretability study of language confusion.<n>We show that confusion points (CPs) are central to this phenomenon.<n>We show that editing a small set of critical neurons, identified via comparative analysis with a multilingual-tuned counterpart, substantially mitigates confusion.
  arXiv  Detail & Related papers  (2025-05-22T11:29:17Z)
• Training Neural Networks as Recognizers of Formal Languages [87.06906286950438]
  We train and evaluate neural networks directly as binary classifiers of strings.<n>We provide results on a variety of languages across the Chomsky hierarchy for three neural architectures.<n>Our contributions will facilitate theoretically sound empirical testing of language recognition claims in future work.
  arXiv  Detail & Related papers  (2024-11-11T16:33:25Z)
• A Crosslingual Investigation of Conceptualization in 1335 Languages [0.2216657815393579]
  We investigate differences in conceptualization across 1,335 languages by aligning concepts in a parallel corpus. We propose Conceptualizer, a method that creates a bipartite directed alignment graph between source language concepts and sets of target language strings. In a detailed linguistic analysis across all languages for one concept (bird') and an evaluation on gold standard data for 32 Swadesh concepts, we show that Conceptualizer has good alignment accuracy.
  arXiv  Detail & Related papers  (2023-05-15T09:27:34Z)
• Feature-rich multiplex lexical networks reveal mental strategies of early language learning [0.7111443975103329]
  We introduce FEature-Rich MUltiplex LEXical (FERMULEX) networks. Similarities model heterogenous word associations across semantic/syntactic/phonological aspects of knowledge. Words are enriched with multi-dimensional feature embeddings including frequency, age of acquisition, length and polysemy.
  arXiv  Detail & Related papers  (2022-01-13T16:44:51Z)
• A Massively Multilingual Analysis of Cross-linguality in Shared Embedding Space [61.18554842370824]
  In cross-lingual language models, representations for many different languages live in the same space. We compute a task-based measure of cross-lingual alignment in the form of bitext retrieval performance. We examine a range of linguistic, quasi-linguistic, and training-related features as potential predictors of these alignment metrics.
  arXiv  Detail & Related papers  (2021-09-13T21:05:37Z)
• Neural Combinatory Constituency Parsing [12.914521751805658]
  Our models decompose the bottom-up parsing process into 1) classification of tags, labels, and binary orientations or chunks and 2) vector composition based on the computed orientations or chunks. The binary model achieves an F1 score of 92.54 on Penn Treebank, speeding at 1327.2 sents/sec. Both the models with XLNet provide near state-of-the-art accuracies for English.
  arXiv  Detail & Related papers  (2021-06-12T05:14:16Z)
• Explicit Alignment Objectives for Multilingual Bidirectional Encoders [111.65322283420805]
  We present a new method for learning multilingual encoders, AMBER (Aligned Multilingual Bi-directional EncodeR) AMBER is trained on additional parallel data using two explicit alignment objectives that align the multilingual representations at different granularities. Experimental results show that AMBER obtains gains of up to 1.1 average F1 score on sequence tagging and up to 27.3 average accuracy on retrieval over the XLMR-large model.
  arXiv  Detail & Related papers  (2020-10-15T18:34:13Z)
• Pre-training Multilingual Neural Machine Translation by Leveraging Alignment Information [72.2412707779571]
  mRASP is an approach to pre-train a universal multilingual neural machine translation model. We carry out experiments on 42 translation directions across a diverse setting, including low, medium, rich resource, and as well as transferring to exotic language pairs.
  arXiv  Detail & Related papers  (2020-10-07T03:57:54Z)
• Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translation [81.7786241489002]
  Massively multilingual models for neural machine translation (NMT) are theoretically attractive, but often underperform bilingual models and deliver poor zero-shot translations. We argue that multilingual NMT requires stronger modeling capacity to support language pairs with varying typological characteristics. We propose random online backtranslation to enforce the translation of unseen training language pairs.
  arXiv  Detail & Related papers  (2020-04-24T17:21:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.