TADA: Task-Agnostic Dialect Adapters for English

• URL: http://arxiv.org/abs/2305.16651v1
• Date: Fri, 26 May 2023 05:45:03 GMT
• Title: TADA: Task-Agnostic Dialect Adapters for English
• Authors: Will Held, Caleb Ziems, Diyi Yang
• Abstract summary: Large Language Models fail at a higher rate for speakers of English dialects other than Standard American English (SAE) We introduce a simple yet effective method for task-agnostic dialect adaptation by aligning non-SAE dialects using adapters and composing them with task-specific adapters from SAE.
• Score: 44.39195676661607
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large Language Models, the dominant starting point for Natural Language Processing (NLP) applications, fail at a higher rate for speakers of English dialects other than Standard American English (SAE). Prior work addresses this using task-specific data or synthetic data augmentation, both of which require intervention for each dialect and task pair. This poses a scalability issue that prevents the broad adoption of robust dialectal English NLP. We introduce a simple yet effective method for task-agnostic dialect adaptation by aligning non-SAE dialects using adapters and composing them with task-specific adapters from SAE. Task-Agnostic Dialect Adapters (TADA) improve dialectal robustness on 4 dialectal variants of the GLUE benchmark without task-specific supervision.

Related papers

• An Initial Investigation of Language Adaptation for TTS Systems under Low-resource Scenarios [76.11409260727459]
  This paper explores the language adaptation capability of ZMM-TTS, a recent SSL-based multilingual TTS system. We demonstrate that the similarity in phonetics between the pre-training and target languages, as well as the language category, affects the target language's adaptation performance.
  arXiv  Detail & Related papers  (2024-06-13T08:16:52Z)
• AdaMergeX: Cross-Lingual Transfer with Large Language Models via Adaptive Adapter Merging [96.39773974044041]
  Cross-lingual transfer is an effective alternative to the direct fine-tuning on target tasks in specific languages. We propose a new cross-lingual transfer method called $textttAdaMergeX$ that utilizes adaptive adapter merging. Our empirical results demonstrate that our approach yields new and effective cross-lingual transfer, outperforming existing methods across all settings.
  arXiv  Detail & Related papers  (2024-02-29T07:11:24Z)
• What Do Dialect Speakers Want? A Survey of Attitudes Towards Language Technology for German Dialects [60.8361859783634]
  We survey speakers of dialects and regional languages related to German. We find that respondents are especially in favour of potential NLP tools that work with dialectal input.
  arXiv  Detail & Related papers  (2024-02-19T09:15:28Z)
• Task-Agnostic Low-Rank Adapters for Unseen English Dialects [52.88554155235167]
  Large Language Models (LLMs) are trained on corpora disproportionally weighted in favor of Standard American English. By disentangling dialect-specific and cross-dialectal information, HyperLoRA improves generalization to unseen dialects in a task-agnostic fashion.
  arXiv  Detail & Related papers  (2023-11-02T01:17:29Z)
• Simple yet Effective Code-Switching Language Identification with Multitask Pre-Training and Transfer Learning [0.7242530499990028]
  Code-switching is the linguistics phenomenon where in casual settings, multilingual speakers mix words from different languages in one utterance. We propose two novel approaches toward improving language identification accuracy on an English-Mandarin child-directed speech dataset. Our best model achieves a balanced accuracy of 0.781 on a real English-Mandarin code-switching child-directed speech corpus and outperforms the previous baseline by 55.3%.
  arXiv  Detail & Related papers  (2023-05-31T11:43:16Z)
• DADA: Dialect Adaptation via Dynamic Aggregation of Linguistic Rules [64.93179829965072]
  DADA is a modular approach to imbue SAE-trained models with multi-dialectal robustness. We show that DADA is effective for both single task and instruction fine language models.
  arXiv  Detail & Related papers  (2023-05-22T18:43:31Z)
• Lightweight Adapter Tuning for Multilingual Speech Translation [47.89784337058167]
  Adapter modules were recently introduced as an efficient alternative to fine-tuning in NLP. This paper proposes a comprehensive analysis of adapters for multilingual speech translation.
  arXiv  Detail & Related papers  (2021-06-02T20:51:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.