Neural Unsupervised Reconstruction of Protolanguage Word Forms

• URL: http://arxiv.org/abs/2211.08684v1
• Date: Wed, 16 Nov 2022 05:38:51 GMT
• Title: Neural Unsupervised Reconstruction of Protolanguage Word Forms
• Authors: Andre He, Nicholas Tomlin, Dan Klein
• Abstract summary: We present a state-of-the-art neural approach to the unsupervised reconstruction of ancient word forms. We extend this work with neural models that can capture more complicated phonological and morphological changes.
• Score: 34.66200889614538
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present a state-of-the-art neural approach to the unsupervised reconstruction of ancient word forms. Previous work in this domain used expectation-maximization to predict simple phonological changes between ancient word forms and their cognates in modern languages. We extend this work with neural models that can capture more complicated phonological and morphological changes. At the same time, we preserve the inductive biases from classical methods by building monotonic alignment constraints into the model and deliberately underfitting during the maximization step. We evaluate our performance on the task of reconstructing Latin from a dataset of cognates across five Romance languages, achieving a notable reduction in edit distance from the target word forms compared to previous methods.

Related papers

• Improved Neural Protoform Reconstruction via Reflex Prediction [11.105362395278142]
  We argue that not only should protoforms be inferable from cognate sets (sets of related reflexes) but the reflexes should also be inferable from the protoforms. We propose a system in which candidate protoforms from a reconstruction model are reranked by a reflex prediction model.
  arXiv  Detail & Related papers  (2024-03-27T17:13:38Z)
• Pairing Orthographically Variant Literary Words to Standard Equivalents Using Neural Edit Distance Models [0.0]
  We present a novel corpus consisting of orthographically variant words found in works of 19th century U.S. literature annotated with their corresponding "standard" word pair. We train a set of neural edit distance models to pair these variants with their standard forms, and compare the performance of these models to the performance of a set of neural edit distance models trained on a corpus of orthographic errors made by L2 English learners.
  arXiv  Detail & Related papers  (2024-01-26T18:49:34Z)
• Representing and Computing Uncertainty in Phonological Reconstruction [5.284425534494986]
  Despite the inherently fuzzy nature of reconstructions in historical linguistics, most scholars do not represent their uncertainty when proposing proto-forms. We present a new framework that allows for the representation of uncertainty in linguistic reconstruction and also includes a workflow for the computation of fuzzy reconstructions from linguistic data.
  arXiv  Detail & Related papers  (2023-10-19T13:27:42Z)
• Cognate Transformer for Automated Phonological Reconstruction and Cognate Reflex Prediction [4.609569810881602]
  We adapt MSA Transformer, a protein language model, to the problem of automated phonological reconstruction. MSA Transformer trains on multiple sequence alignments as input and is, thus, apt for application on aligned cognate words. We also apply the model on another associated task, namely, cognate reflex prediction, where a reflex word in a daughter language is predicted based on cognate words from other daughter languages.
  arXiv  Detail & Related papers  (2023-10-11T13:34:22Z)
• Unsupervised Lexical Substitution with Decontextualised Embeddings [48.00929769805882]
  We propose a new unsupervised method for lexical substitution using pre-trained language models. Our method retrieves substitutes based on the similarity of contextualised and decontextualised word embeddings. We conduct experiments in English and Italian, and show that our method substantially outperforms strong baselines.
  arXiv  Detail & Related papers  (2022-09-17T03:51:47Z)
• SLM: Learning a Discourse Language Representation with Sentence Unshuffling [53.42814722621715]
  We introduce Sentence-level Language Modeling, a new pre-training objective for learning a discourse language representation. We show that this feature of our model improves the performance of the original BERT by large margins.
  arXiv  Detail & Related papers  (2020-10-30T13:33:41Z)
• On Long-Tailed Phenomena in Neural Machine Translation [50.65273145888896]
  State-of-the-art Neural Machine Translation (NMT) models struggle with generating low-frequency tokens. We propose a new loss function, the Anti-Focal loss, to better adapt model training to the structural dependencies of conditional text generation. We show the efficacy of the proposed technique on a number of Machine Translation (MT) datasets, demonstrating that it leads to significant gains over cross-entropy.
  arXiv  Detail & Related papers  (2020-10-10T07:00:57Z)
• Neural Baselines for Word Alignment [0.0]
  We study and evaluate neural models for unsupervised word alignment for four language pairs. We show that neural versions of the IBM-1 and hidden Markov models vastly outperform their discrete counterparts.
  arXiv  Detail & Related papers  (2020-09-28T07:51:03Z)
• Grounded Compositional Outputs for Adaptive Language Modeling [59.02706635250856]
  A language model's vocabulary$-$typically selected before training and permanently fixed later$-$affects its size. We propose a fully compositional output embedding layer for language models. To our knowledge, the result is the first word-level language model with a size that does not depend on the training vocabulary.
  arXiv  Detail & Related papers  (2020-09-24T07:21:14Z)
• Mechanisms for Handling Nested Dependencies in Neural-Network Language Models and Humans [75.15855405318855]
  We studied whether a modern artificial neural network trained with "deep learning" methods mimics a central aspect of human sentence processing. Although the network was solely trained to predict the next word in a large corpus, analysis showed the emergence of specialized units that successfully handled local and long-distance syntactic agreement. We tested the model's predictions in a behavioral experiment where humans detected violations in number agreement in sentences with systematic variations in the singular/plural status of multiple nouns.
  arXiv  Detail & Related papers  (2020-06-19T12:00:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.