Self-Supervised Lexical Representation Learning for Fast, Large-Scale Phylogenetic Inference

• URL: http://arxiv.org/abs/2609.05262v1
• Date: Fri, 04 Sep 2026 15:23:12 GMT
• Title: Self-Supervised Lexical Representation Learning for Fast, Large-Scale Phylogenetic Inference
• Abstract summary: This paper introduces a fully self-supervised contrastive learning framework.<n>It learns lexical representations directly from raw IPA-transcribed wordlists.<n>It infers a global phylogenetic tree of 3,399 language varieties.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Computational phylogenetics has become an essential tool in historical linguistics, yet its application at a global scale remains constrained by two factors: the labor-intensive manual annotation of cognacy judgments required for character-based methods and the substantial computational cost of inference on large datasets. This paper introduces a fully self-supervised contrastive learning framework that learns lexical representations directly from raw IPA-transcribed wordlists, without requiring cognacy annotations, alignments, or additional expert input. The model employs a dual contrastive objective: a word-level loss that organizes phonetically similar forms into a coherent space, and an auxiliary language-level loss that encourages the lexical space to reflect broader phonological properties of languages. From the resulting word representations, pairwise language distances are derived and used to infer a global phylogenetic tree of 3,399 language varieties. The inferred tree achieves a generalized quartet distance (GQD) to the Glottolog reference tree competitive with multiple baselines, while requiring only minutes of computation on a standard notebook GPU. Furthermore, the same representations capture diachronic concept stability: variance in pairwise distances across languages yields stability rankings that correlate significantly with established rankings. Ablation studies confirm that both the language-level objective and the use of phonetic feature vectors improved the inferred trees topology with regards to GQD. The framework thus provides a computationally efficient and fully automatic alternative for large-scale phylogenetic inference and offers a unified representation supporting downstream analyses at both the language and concept level.

Related papers

• Conceptual Networks for Cross-Linguistic Idiomatic Expressions: A Feature-Based Graph Approach [0.0]
  We present an interpretable network-based framework for representing idiomatic and figurative meaning across eight typologically diverse languages.<n>Each expression is annotated with binary conceptual features derived from cognitive-linguistic theory, and pairwise Jaccard similarities define a weighted graph.<n>Community detection reveals that idioms cluster by conceptual schema rather than by language, producing a structure consistent with cognitive-linguistic predictions.
  arXiv  Detail & Related papers  (2026-07-10T16:25:51Z)
• Weave of Formal Thought [51.56484100374058]
  We introduce Weave of Formal Thought (WoFT), a paradigm uniting rigorous syntactic validation with learned structural representations.<n>We present a latent-variable fine-tuning method training the language model to interleave non-terminal grammar symbols directly into generation.<n>For Python, fine-tuning StarCoder2-3B with our RWS objective reduces per-token cross-entropy by 14.3% relative to a text-only SFT baseline.
  arXiv  Detail & Related papers  (2026-06-24T15:58:11Z)
• Beyond cognacy [0.21756081703275998]
  Two fully automated methods are compared to extract phylogenetic signal directly from lexical data.<n>Results show that MSA-based inference yields trees more consistent with linguistic classifications, better predicts typological variation, and provides a clearer phylogenetic signal.
  arXiv  Detail & Related papers  (2025-07-02T06:47:34Z)
• Beyond Chunking: Discourse-Aware Hierarchical Retrieval for Long Document Question Answering [51.7493726399073]
  We present a discourse-aware hierarchical framework to enhance long document question answering.<n>The framework involves three key innovations: specialized discourse parsing for lengthy documents, LLM-based enhancement of discourse relation nodes, and structure-guided hierarchical retrieval.
  arXiv  Detail & Related papers  (2025-05-26T14:45:12Z)
• Evaluating Distributed Representations for Multi-Level Lexical Semantics: A Research Proposal [3.3585951129432323]
  This thesis builds a bridge between computational models and lexical semantics, aiming to complement each other.<n>Modern neural networks (NNs) construct distributed representations by compressing individual words into dense, continuous, high-dimensional vectors.
  arXiv  Detail & Related papers  (2024-06-02T14:08:51Z)
• A Massively Multilingual Analysis of Cross-linguality in Shared Embedding Space [61.18554842370824]
  In cross-lingual language models, representations for many different languages live in the same space. We compute a task-based measure of cross-lingual alignment in the form of bitext retrieval performance. We examine a range of linguistic, quasi-linguistic, and training-related features as potential predictors of these alignment metrics.
  arXiv  Detail & Related papers  (2021-09-13T21:05:37Z)
• Discrete representations in neural models of spoken language [56.29049879393466]
  We compare the merits of four commonly used metrics in the context of weakly supervised models of spoken language. We find that the different evaluation metrics can give inconsistent results.
  arXiv  Detail & Related papers  (2021-05-12T11:02:02Z)
• Infusing Finetuning with Semantic Dependencies [62.37697048781823]
  We show that, unlike syntax, semantics is not brought to the surface by today's pretrained models. We then use convolutional graph encoders to explicitly incorporate semantic parses into task-specific finetuning.
  arXiv  Detail & Related papers  (2020-12-10T01:27:24Z)
• Probing Multilingual BERT for Genetic and Typological Signals [28.360662552057324]
  We probe the layers in multilingual BERT (mBERT) for phylogenetic and geographic language signals across 100 languages. We employ the language distances to infer and evaluate language trees, finding that they are close to the reference family tree in terms of quartet tree distance.
  arXiv  Detail & Related papers  (2020-11-04T00:03:04Z)
• Learning Universal Representations from Word to Sentence [89.82415322763475]
  This work introduces and explores the universal representation learning, i.e., embeddings of different levels of linguistic unit in a uniform vector space. We present our approach of constructing analogy datasets in terms of words, phrases and sentences. We empirically verify that well pre-trained Transformer models incorporated with appropriate training settings may effectively yield universal representation.
  arXiv  Detail & Related papers  (2020-09-10T03:53:18Z)
• Exploiting Syntactic Structure for Better Language Modeling: A Syntactic Distance Approach [78.77265671634454]
  We make use of a multi-task objective, i.e., the models simultaneously predict words as well as ground truth parse trees in a form called "syntactic distances" Experimental results on the Penn Treebank and Chinese Treebank datasets show that when ground truth parse trees are provided as additional training signals, the model is able to achieve lower perplexity and induce trees with better quality.
  arXiv  Detail & Related papers  (2020-05-12T15:35:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.