AdapterHub Playground: Simple and Flexible Few-Shot Learning with Adapters

• URL: http://arxiv.org/abs/2108.08103v1
• Date: Wed, 18 Aug 2021 11:56:01 GMT
• Title: AdapterHub Playground: Simple and Flexible Few-Shot Learning with Adapters
• Authors: Tilman Beck, Bela Bohlender, Christina Viehmann, Vincent Hane, Yanik Adamson, Jaber Khuri, Jonas Brossmann, Jonas Pfeiffer, Iryna Gurevych
• Abstract summary: Open-access dissemination of pretrained language models has led to a democratization of state-of-the-art natural language processing (NLP) research. This also allows people outside of NLP to use such models and adapt them to specific use-cases. In this work, we aim to overcome this gap by providing a tool which allows researchers to leverage pretrained models without writing a single line of code.
• Score: 34.86139827292556
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The open-access dissemination of pretrained language models through online repositories has led to a democratization of state-of-the-art natural language processing (NLP) research. This also allows people outside of NLP to use such models and adapt them to specific use-cases. However, a certain amount of technical proficiency is still required which is an entry barrier for users who want to apply these models to a certain task but lack the necessary knowledge or resources. In this work, we aim to overcome this gap by providing a tool which allows researchers to leverage pretrained models without writing a single line of code. Built upon the parameter-efficient adapter modules for transfer learning, our AdapterHub Playground provides an intuitive interface, allowing the usage of adapters for prediction, training and analysis of textual data for a variety of NLP tasks. We present the tool's architecture and demonstrate its advantages with prototypical use-cases, where we show that predictive performance can easily be increased in a few-shot learning scenario. Finally, we evaluate its usability in a user study. We provide the code and a live interface at https://adapter-hub.github.io/playground.

Related papers

• EduNLP: Towards a Unified and Modularized Library for Educational Resources [78.8523961816045]
  We present a unified, modularized, and extensive library, EduNLP, focusing on educational resource understanding. In the library, we decouple the whole workflow to four key modules with consistent interfaces including data configuration, processing, model implementation, and model evaluation. For the current version, we primarily provide 10 typical models from four categories, and 5 common downstream-evaluation tasks in the education domain on 8 subjects for users' usage.
  arXiv  Detail & Related papers  (2024-06-03T12:45:40Z)
• Make Prompts Adaptable: Bayesian Modeling for Vision-Language Prompt Learning with Data-Dependent Prior [14.232144691524528]
  Recent Vision-Language Pretrained models have become the backbone for many downstream tasks. MLE training can lead the context vector to over-fit dominant image features in the training data. This paper presents a Bayesian-based framework of prompt learning, which could alleviate the overfitting issues on few-shot learning application.
  arXiv  Detail & Related papers  (2024-01-09T10:15:59Z)
• Adapters: A Unified Library for Parameter-Efficient and Modular Transfer Learning [109.25673110120906]
  We introduce Adapters, an open-source library that unifies parameter-efficient and modular transfer learning in large language models. By integrating 10 diverse adapter methods into a unified interface, Adapters offers ease of use and flexible configuration.
  arXiv  Detail & Related papers  (2023-11-18T13:53:26Z)
• Adaptive Parameterization of Deep Learning Models for Federated Learning [85.82002651944254]
  Federated Learning offers a way to train deep neural networks in a distributed fashion. It incurs a communication overhead as the model parameters or gradients need to be exchanged regularly during training. In this paper, we propose to utilise parallel Adapters for Federated Learning.
  arXiv  Detail & Related papers  (2023-02-06T17:30:33Z)
• CHAPTER: Exploiting Convolutional Neural Network Adapters for Self-supervised Speech Models [62.60723685118747]
  Self-supervised learning (SSL) is a powerful technique for learning representations from unlabeled data. We propose an efficient tuning method specifically designed for SSL speech model, by applying CNN adapters at the feature extractor. We empirically found that adding CNN to the feature extractor can help the adaptation on emotion and speaker tasks.
  arXiv  Detail & Related papers  (2022-12-01T08:50:12Z)
• On The Cross-Modal Transfer from Natural Language to Code through Adapter Modules [0.0]
  We explore the knowledge transfer using adapters in software engineering. Three programming languages, C/C++, Python, and Java, are studied along with extensive experiments on the best setup used for adapters. Our results can open new directions to build smaller models for more software engineering tasks.
  arXiv  Detail & Related papers  (2022-04-19T04:18:02Z)
• Exploiting Adapters for Cross-lingual Low-resource Speech Recognition [52.40623653290499]
  Cross-lingual speech adaptation aims to solve the problem of leveraging multiple rich-resource languages to build models for a low-resource target language. We propose adapters to investigate the performance of multiple adapters for parameter-efficient cross-lingual speech adaptation.
  arXiv  Detail & Related papers  (2021-05-18T08:30:37Z)
• EXPATS: A Toolkit for Explainable Automated Text Scoring [2.299617836036273]
  We present EXPATS, an open-source framework to allow users to develop and experiment with different ATS models quickly. The toolkit also provides seamless integration with the Language Interpretability Tool (LIT) so that one can interpret and visualize models and their predictions.
  arXiv  Detail & Related papers  (2021-04-07T19:29:06Z)
• AdapterHub: A Framework for Adapting Transformers [148.6877231725939]
  AdapterHub is a framework that allows dynamic "stitching-in" of pre-trained adapters for different tasks and languages. Our framework enables scalable and easy access to sharing of task-specific models.
  arXiv  Detail & Related papers  (2020-07-15T15:56:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.