Adapter-TST: A Parameter Efficient Method for Multiple-Attribute Text Style Transfer

• URL: http://arxiv.org/abs/2305.05945v1
• Date: Wed, 10 May 2023 07:33:36 GMT
• Title: Adapter-TST: A Parameter Efficient Method for Multiple-Attribute Text Style Transfer
• Authors: Zhiqiang Hu, Roy Ka-Wei Lee, Nancy F. Chen
• Abstract summary: AdapterTST is a framework that freezes the pre-trained model's original parameters and enables the development of a multiple-attribute text style transfer model. We evaluate the proposed model on both traditional sentiment transfer and multiple-attribute transfer tasks.
• Score: 29.67331801326995
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Adapting a large language model for multiple-attribute text style transfer via fine-tuning can be challenging due to the significant amount of computational resources and labeled data required for the specific task. In this paper, we address this challenge by introducing AdapterTST, a framework that freezes the pre-trained model's original parameters and enables the development of a multiple-attribute text style transfer model. Using BART as the backbone model, Adapter-TST utilizes different neural adapters to capture different attribute information, like a plug-in connected to BART. Our method allows control over multiple attributes, like sentiment, tense, voice, etc., and configures the adapters' architecture to generate multiple outputs respected to attributes or compositional editing on the same sentence. We evaluate the proposed model on both traditional sentiment transfer and multiple-attribute transfer tasks. The experiment results demonstrate that Adapter-TST outperforms all the state-of-the-art baselines with significantly lesser computational resources. We have also empirically shown that each adapter is able to capture specific stylistic attributes effectively and can be configured to perform compositional editing.

Related papers

• Att-Adapter: A Robust and Precise Domain-Specific Multi-Attributes T2I Diffusion Adapter via Conditional Variational Autoencoder [11.392007197036525]
  We introduce the Attribute (Att) Adapter, a novel plug-and-play module designed to enable fine-grained, multi-attributes control in pretrained diffusion models. Att-Adapter is flexible, requiring no paired synthetic data for training, and is easily scalable to multiple attributes within a single model.
  arXiv  Detail & Related papers  (2025-03-15T01:06:34Z)
• Towards Lightweight, Adaptive and Attribute-Aware Multi-Aspect Controllable Text Generation with Large Language Models [40.54453001537357]
  Multi-aspect controllable text generation aims to control text generation in attributes from multiple aspects. Supervised fine-tuning methods are often employed for this task due to their simplicity and effectiveness. We propose a lightweight, adaptive and attribute-aware framework for multi-aspect controllable text generation.
  arXiv  Detail & Related papers  (2025-02-19T06:56:02Z)
• UNIC-Adapter: Unified Image-instruction Adapter with Multi-modal Transformer for Image Generation [64.8341372591993]
  We propose a new approach to unify controllable generation within a single framework. Specifically, we propose the unified image-instruction adapter (UNIC-Adapter) built on the Multi-Modal-Diffusion Transformer architecture. Our UNIC-Adapter effectively extracts multi-modal instruction information by incorporating both conditional images and task instructions.
  arXiv  Detail & Related papers  (2024-12-25T15:19:02Z)
• iConFormer: Dynamic Parameter-Efficient Tuning with Input-Conditioned Adaptation [15.97351561456467]
  In this paper, we propose a novel PEFT approach, input-Conditioned transFormer, termed iConFormer. We introduce an input-Conditioned Network (iCoN) in the dynamic adapter that enables instance-level feature transformation. To be specific, iCoN generates channel-wise convolutional kernels for each feature and transform it using adaptive convolution process to effectively capture task-specific and fine-grained details tailor to downstream tasks.
  arXiv  Detail & Related papers  (2024-09-04T16:06:23Z)
• MV-Adapter: Multimodal Video Transfer Learning for Video Text Retrieval [60.454321238910474]
  State-of-the-art video-text retrieval methods typically involve fully fine-tuning a pre-trained model on specific datasets. We present our pioneering work that enables parameter-efficient VTR using a pre-trained model. We propose a new method dubbed Multimodal Video Adapter (MV-Adapter) for efficiently transferring the knowledge in the pre-trained CLIP from image-text to video-text.
  arXiv  Detail & Related papers  (2023-01-19T03:42:56Z)
• ManiCLIP: Multi-Attribute Face Manipulation from Text [104.30600573306991]
  We present a novel multi-attribute face manipulation method based on textual descriptions. Our method generates natural manipulated faces with minimal text-irrelevant attribute editing.
  arXiv  Detail & Related papers  (2022-10-02T07:22:55Z)
• AdaMix: Mixture-of-Adapter for Parameter-efficient Tuning of Large Language Models [119.7093605087114]
  Fine-tuning large-scale pre-trained language models to downstream tasks require updating hundreds of millions of parameters. This not only increases the serving cost to store a large copy of the model weights for every task, but also exhibits instability during few-shot task adaptation. We introduce a new mechanism to improve adapter capacity without increasing parameters or computational cost by two key techniques.
  arXiv  Detail & Related papers  (2022-05-24T23:41:22Z)
• Adaptable Adapters [74.65986170056945]
  State-of-the-art pretrained NLP models contain a hundred million to trillion parameters. Adaptable adapters contain different activation functions for different layers and different input data. We show that adaptable adapters achieve on-par performances with the standard adapter architecture.
  arXiv  Detail & Related papers  (2022-05-03T14:59:27Z)
• Parameter-efficient Multi-task Fine-tuning for Transformers via Shared Hypernetworks [37.2958914602899]
  We show that we can learn adapter parameters for all layers and tasks by generating them using shared hypernetworks. Experiments on the well-known GLUE benchmark show improved performance in multi-task learning while adding only 0.29% parameters per task.
  arXiv  Detail & Related papers  (2021-06-08T16:16:40Z)
• Lightweight Adapter Tuning for Multilingual Speech Translation [47.89784337058167]
  Adapter modules were recently introduced as an efficient alternative to fine-tuning in NLP. This paper proposes a comprehensive analysis of adapters for multilingual speech translation.
  arXiv  Detail & Related papers  (2021-06-02T20:51:42Z)
• AdapterHub: A Framework for Adapting Transformers [148.6877231725939]
  AdapterHub is a framework that allows dynamic "stitching-in" of pre-trained adapters for different tasks and languages. Our framework enables scalable and easy access to sharing of task-specific models.
  arXiv  Detail & Related papers  (2020-07-15T15:56:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.