CMULAB: An Open-Source Framework for Training and Deployment of Natural Language Processing Models

• URL: http://arxiv.org/abs/2404.02408v1
• Date: Wed, 3 Apr 2024 02:21:46 GMT
• Title: CMULAB: An Open-Source Framework for Training and Deployment of Natural Language Processing Models
• Authors: Zaid Sheikh, Antonios Anastasopoulos, Shruti Rijhwani, Lindia Tjuatja, Robbie Jimerson, Graham Neubig,
• Abstract summary: This paper introduces the CMU Linguistic Linguistic Backend, an open-source framework that simplifies model deployment and continuous human-in-the-loop fine-tuning of NLP models. CMULAB enables users to leverage the power of multilingual models to quickly adapt and extend existing tools for speech recognition, OCR, translation, and syntactic analysis to new languages.
• Score: 59.91221728187576
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Effectively using Natural Language Processing (NLP) tools in under-resourced languages requires a thorough understanding of the language itself, familiarity with the latest models and training methodologies, and technical expertise to deploy these models. This could present a significant obstacle for language community members and linguists to use NLP tools. This paper introduces the CMU Linguistic Annotation Backend, an open-source framework that simplifies model deployment and continuous human-in-the-loop fine-tuning of NLP models. CMULAB enables users to leverage the power of multilingual models to quickly adapt and extend existing tools for speech recognition, OCR, translation, and syntactic analysis to new languages, even with limited training data. We describe various tools and APIs that are currently available and how developers can easily add new models/functionality to the framework. Code is available at https://github.com/neulab/cmulab along with a live demo at https://cmulab.dev

Related papers

• Towards a More Inclusive AI: Progress and Perspectives in Large Language Model Training for the Sámi Language [7.289015788793582]
  This work focuses on increasing technological participation for the S'ami language. We draw the attention of the ML community towards the language modeling problem of Ultra Low Resource (ULR) languages. We have compiled the available S'ami language resources from the web to create a clean dataset for training language models.
  arXiv  Detail & Related papers  (2024-05-09T13:54:22Z)
• Lemur: Harmonizing Natural Language and Code for Language Agents [107.44942854856802]
  We introduce Lemur and Lemur-Chat, open-source language models optimized for both natural language and coding capabilities. Our models achieve state-of-the-art averaged performance across diverse text and coding benchmarks. The harmonization between natural and programming languages enables Lemur-Chat to significantly narrow the gap with proprietary models on agent abilities.
  arXiv  Detail & Related papers  (2023-10-10T17:57:45Z)
• Improving Language Plasticity via Pretraining with Active Forgetting [63.36484652568976]
  We propose to use an active forgetting mechanism during pretraining, as a simple way of creating PLMs that can quickly adapt to new languages. Experiments with RoBERTa show that models pretrained with our forgetting mechanism demonstrate faster convergence during language adaptation.
  arXiv  Detail & Related papers  (2023-07-03T17:12:44Z)
• Soft Language Clustering for Multilingual Model Pre-training [57.18058739931463]
  We propose XLM-P, which contextually retrieves prompts as flexible guidance for encoding instances conditionally. Our XLM-P enables (1) lightweight modeling of language-invariant and language-specific knowledge across languages, and (2) easy integration with other multilingual pre-training methods.
  arXiv  Detail & Related papers  (2023-06-13T08:08:08Z)
• AfroLM: A Self-Active Learning-based Multilingual Pretrained Language Model for 23 African Languages [0.021987601456703476]
  We present AfroLM, a multilingual language model pretrained from scratch on 23 African languages. AfroLM is pretrained on a dataset 14x smaller than existing baselines. It is able to generalize well across various domains.
  arXiv  Detail & Related papers  (2022-11-07T02:15:25Z)
• Generalizing Multimodal Pre-training into Multilingual via Language Acquisition [54.69707237195554]
  English-based Vision-Language Pre-training has achieved great success in various downstream tasks. Some efforts have been taken to generalize this success to non-English languages through Multilingual Vision-Language Pre-training. We propose a textbfMultitextbfLingual textbfAcquisition (MLA) framework that can easily generalize a monolingual Vision-Language Pre-training model into multilingual.
  arXiv  Detail & Related papers  (2022-05-29T08:53:22Z)
• LaoPLM: Pre-trained Language Models for Lao [3.2146309563776416]
  Pre-trained language models (PLMs) can capture different levels of concepts in context and hence generate universal language representations. Although PTMs have been widely used in most NLP applications, it is under-represented in Lao NLP research. We construct a text classification dataset to alleviate the resource-scare situation of the Lao language. We present the first transformer-based PTMs for Lao with four versions: BERT-small, BERT-base, ELECTRA-small and ELECTRA-base, and evaluate it over two downstream tasks: part-of-speech tagging and text classification.
  arXiv  Detail & Related papers  (2021-10-12T11:13:07Z)
• UNKs Everywhere: Adapting Multilingual Language Models to New Scripts [103.79021395138423]
  Massively multilingual language models such as multilingual BERT (mBERT) and XLM-R offer state-of-the-art cross-lingual transfer performance on a range of NLP tasks. Due to their limited capacity and large differences in pretraining data, there is a profound performance gap between resource-rich and resource-poor target languages. We propose novel data-efficient methods that enable quick and effective adaptation of pretrained multilingual models to such low-resource languages and unseen scripts.
  arXiv  Detail & Related papers  (2020-12-31T11:37:28Z)
• Evaluating Cross-Lingual Transfer Learning Approaches in Multilingual Conversational Agent Models [1.52292571922932]
  We propose a general multilingual model framework for Natural Language Understanding (NLU) models. We show that these multilingual models can reach same or better performance compared to monolingual models across language-specific test data.
  arXiv  Detail & Related papers  (2020-12-07T17:14:52Z)
• WikiBERT models: deep transfer learning for many languages [1.3455090151301572]
  We introduce a simple, fully automated pipeline for creating languagespecific BERT models from Wikipedia data. We assess the merits of these models using the state-of-the-art UDify on Universal Dependencies data.
  arXiv  Detail & Related papers  (2020-06-02T11:57:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.