Prompt and circumstance: A word-by-word LLM prompting approach to interlinear glossing for low-resource languages

• URL: http://arxiv.org/abs/2502.09778v2
• Date: Fri, 21 Mar 2025 22:24:06 GMT
• Title: Prompt and circumstance: A word-by-word LLM prompting approach to interlinear glossing for low-resource languages
• Authors: Micha Elsner, David Liu,
• Abstract summary: We investigate the effectiveness of a retrieval-based LLM prompting approach to glossing, applied to the seven languages from the SIGMORPHON 2023 shared task.<n>Our system beats the BERT-based shared task baseline for every language in the morpheme-level score category.<n>In a case study on Tsez, we ask the LLM to automatically create and follow linguistic instructions, reducing errors on a confusing grammatical feature.
• Score: 6.4977738682502295
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Partly automated creation of interlinear glossed text (IGT) has the potential to assist in linguistic documentation. We argue that LLMs can make this process more accessible to linguists because of their capacity to follow natural-language instructions. We investigate the effectiveness of a retrieval-based LLM prompting approach to glossing, applied to the seven languages from the SIGMORPHON 2023 shared task. Our system beats the BERT-based shared task baseline for every language in the morpheme-level score category, and we show that a simple 3-best oracle has higher word-level scores than the challenge winner (a tuned sequence model) in five languages. In a case study on Tsez, we ask the LLM to automatically create and follow linguistic instructions, reducing errors on a confusing grammatical feature. Our results thus demonstrate the potential contributions which LLMs can make in interactive systems for glossing, both in making suggestions to human annotators and following directions.

Related papers

• Self-Correction Makes LLMs Better Parsers [19.20952673157709]
  Large language models (LLMs) have achieved remarkable success across various natural language processing (NLP) tasks. Recent studies suggest that they still face challenges in performing fundamental NLP tasks essential for deep language understanding. We propose a self-correction method that leverages grammar rules from existing treebanks to guide LLMs in correcting previous errors.
  arXiv  Detail & Related papers  (2025-04-19T03:50:59Z)
• Linguistic Blind Spots of Large Language Models [14.755831733659699]
  We study the performance of recent large language models (LLMs) on linguistic annotation tasks. We find that recent LLMs show limited efficacy in addressing linguistic queries and often struggle with linguistically complex inputs. Our results provide insights to inform future advancements in LLM design and development.
  arXiv  Detail & Related papers  (2025-03-25T01:47:13Z)
• Can LLMs Help Create Grammar?: Automating Grammar Creation for Endangered Languages with In-Context Learning [0.0]
  This paper explores how Large Language Models (LLMs) can assist in generating grammatical information for low-resource languages with limited amount of data.<n>Our methodology involves organising the existing linguistic data and prompting to efficiently enable to generate formal XLE grammar.<n>This study highlights the potential of LLMs to enhance language documentation efforts, providing a cost-effective solution for generating linguistic data and contributing to the preservation of endangered languages.
  arXiv  Detail & Related papers  (2024-12-14T20:43:12Z)
• Think Carefully and Check Again! Meta-Generation Unlocking LLMs for Low-Resource Cross-Lingual Summarization [108.6908427615402]
  Cross-lingual summarization ( CLS) aims to generate a summary for the source text in a different target language. Currently, instruction-tuned large language models (LLMs) excel at various English tasks. Recent studies have shown that LLMs' performance on CLS tasks remains unsatisfactory even with few-shot settings.
  arXiv  Detail & Related papers  (2024-10-26T00:39:44Z)
• Learning-From-Mistakes Prompting for Indigenous Language Translation [3.7790255156708397]
  This paper presents techniques to improve extremely low-resourced indigenous language translations. Our approaches are grounded in the use of a datastore consisting of a limited number of parallel translation examples. We harness the potential of LLMs and in-context learning techniques in such a setting for using LLMs as universal translators.
  arXiv  Detail & Related papers  (2024-07-18T09:41:20Z)
• PhonologyBench: Evaluating Phonological Skills of Large Language Models [57.80997670335227]
  Phonology, the study of speech's structure and pronunciation rules, is a critical yet often overlooked component in Large Language Model (LLM) research. We present PhonologyBench, a novel benchmark consisting of three diagnostic tasks designed to explicitly test the phonological skills of LLMs. We observe a significant gap of 17% and 45% on Rhyme Word Generation and Syllable counting, respectively, when compared to humans.
  arXiv  Detail & Related papers  (2024-04-03T04:53:14Z)
• How do Large Language Models Handle Multilingualism? [81.15060972112563]
  This study explores how large language models (LLMs) handle multilingualism. LLMs initially understand the query, converting multilingual inputs into English for task-solving. In the intermediate layers, they employ English for thinking and incorporate multilingual knowledge with self-attention and feed-forward structures.
  arXiv  Detail & Related papers  (2024-02-29T02:55:26Z)
• Language-Specific Neurons: The Key to Multilingual Capabilities in Large Language Models [117.20416338476856]
  Large language models (LLMs) demonstrate remarkable multilingual capabilities without being pre-trained on specially curated multilingual parallel corpora. We propose a novel detection method, language activation probability entropy (LAPE), to identify language-specific neurons within LLMs. Our findings indicate that LLMs' proficiency in processing a particular language is predominantly due to a small subset of neurons.
  arXiv  Detail & Related papers  (2024-02-26T09:36:05Z)
• If LLM Is the Wizard, Then Code Is the Wand: A Survey on How Code Empowers Large Language Models to Serve as Intelligent Agents [81.60906807941188]
  Large language models (LLMs) are trained on a combination of natural language and formal language (code) Code translates high-level goals into executable steps, featuring standard syntax, logical consistency, abstraction, and modularity.
  arXiv  Detail & Related papers  (2024-01-01T16:51:20Z)
• Democratizing LLMs for Low-Resource Languages by Leveraging their English Dominant Abilities with Linguistically-Diverse Prompts [75.33019401706188]
  Large language models (LLMs) are known to effectively perform tasks by simply observing few exemplars. We propose to assemble synthetic exemplars from a diverse set of high-resource languages to prompt the LLMs to translate from any language into English. Our unsupervised prompting method performs on par with supervised few-shot learning in LLMs of different sizes for translations between English and 13 Indic and 21 African low-resource languages.
  arXiv  Detail & Related papers  (2023-06-20T08:27:47Z)
• Efficiently Aligned Cross-Lingual Transfer Learning for Conversational Tasks using Prompt-Tuning [98.60739735409243]
  Cross-lingual transfer of language models trained on high-resource languages like English has been widely studied for many NLP tasks. We introduce XSGD for cross-lingual alignment pretraining, a parallel and large-scale multilingual conversation dataset. To facilitate aligned cross-lingual representations, we develop an efficient prompt-tuning-based method for learning alignment prompts.
  arXiv  Detail & Related papers  (2023-04-03T18:46:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.