Domain-Oriented Prefix-Tuning: Towards Efficient and Generalizable Fine-tuning for Zero-Shot Dialogue Summarization

• URL: http://arxiv.org/abs/2204.04362v1
• Date: Sat, 9 Apr 2022 02:28:22 GMT
• Title: Domain-Oriented Prefix-Tuning: Towards Efficient and Generalizable Fine-tuning for Zero-Shot Dialogue Summarization
• Authors: Lulu Zhao, Fujia Zheng, Weihao Zeng, Keqing He, Weiran Xu, Huixing Jiang, Wei Wu, Yanan Wu
• Abstract summary: We propose an efficient and generalizable Domain-Oriented Prefix-tuning model to alleviate domain entanglement. We conduct zero-shot experiments and build domain adaptation benchmarks on two multi-domain dialogue summarization datasets.
• Score: 29.700113636257544
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The most advanced abstractive dialogue summarizers lack generalization ability on new domains and the existing researches for domain adaptation in summarization generally rely on large-scale pre-trainings. To explore the lightweight fine-tuning methods for domain adaptation of dialogue summarization, in this paper, we propose an efficient and generalizable Domain-Oriented Prefix-tuning model, which utilizes a domain word initialized prefix module to alleviate domain entanglement and adopts discrete prompts to guide the model to focus on key contents of dialogues and enhance model generalization. We conduct zero-shot experiments and build domain adaptation benchmarks on two multi-domain dialogue summarization datasets, TODSum and QMSum. Adequate experiments and qualitative analysis prove the effectiveness of our methods.

Related papers

• Schema Augmentation for Zero-Shot Domain Adaptation in Dialogue State Tracking [16.67185296899117]
  Current large language model approaches for zero-shot domain adaptation rely on prompting to introduce knowledge pertaining to the target domains. In this work, we devise a novel data augmentation approach, Augmentation, that improves the zero-shot domain adaptation of language models through fine-tuning. Experiments on MultiWOZ and SpokenWOZ showed that the proposed approach resulted in a substantial improvement over the baseline.
  arXiv  Detail & Related papers  (2024-10-31T18:57:59Z)
• DomainSum: A Hierarchical Benchmark for Fine-Grained Domain Shift in Abstractive Text Summarization [1.7009235747761653]
  DomainSum is a hierarchical benchmark designed to capture fine-grained domain shifts in abstractive summarization. We categorize these shifts into three levels: genre, style, and topic, and demonstrate through comprehensive benchmark analysis that they follow a hierarchical structure.
  arXiv  Detail & Related papers  (2024-10-21T06:55:35Z)
• Learning to Generalize Unseen Domains via Multi-Source Meta Learning for Text Classification [71.08024880298613]
  We study the multi-source Domain Generalization of text classification. We propose a framework to use multiple seen domains to train a model that can achieve high accuracy in an unseen domain.
  arXiv  Detail & Related papers  (2024-09-20T07:46:21Z)
• Unified Language-driven Zero-shot Domain Adaptation [55.64088594551629]
  Unified Language-driven Zero-shot Domain Adaptation (ULDA) is a novel task setting. It enables a single model to adapt to diverse target domains without explicit domain-ID knowledge.
  arXiv  Detail & Related papers  (2024-04-10T16:44:11Z)
• Role Prompting Guided Domain Adaptation with General Capability Preserve for Large Language Models [55.51408151807268]
  When tailored to specific domains, Large Language Models (LLMs) tend to experience catastrophic forgetting. crafting a versatile model for multiple domains simultaneously often results in a decline in overall performance. We present the RolE Prompting Guided Multi-Domain Adaptation (REGA) strategy.
  arXiv  Detail & Related papers  (2024-03-05T08:22:41Z)
• Memory-Efficient Prompt Tuning for Incremental Histopathology Classification [69.46798702300042]
  We present a memory-efficient prompt tuning framework to cultivate model generalization potential in economical memory cost. We have extensively evaluated our framework with two histopathology tasks, i.e., breast cancer metastasis classification and epithelium-stroma tissue classification.
  arXiv  Detail & Related papers  (2024-01-22T03:24:45Z)
• Cross Contrasting Feature Perturbation for Domain Generalization [11.863319505696184]
  Domain generalization aims to learn a robust model from source domains that generalize well on unseen target domains. Recent studies focus on generating novel domain samples or features to diversify distributions complementary to source domains. We propose an online one-stage Cross Contrasting Feature Perturbation framework to simulate domain shift.
  arXiv  Detail & Related papers  (2023-07-24T03:27:41Z)
• TAL: Two-stream Adaptive Learning for Generalizable Person Re-identification [115.31432027711202]
  We argue that both domain-specific and domain-invariant features are crucial for improving the generalization ability of re-id models. We name two-stream adaptive learning (TAL) to simultaneously model these two kinds of information. Our framework can be applied to both single-source and multi-source domain generalization tasks.
  arXiv  Detail & Related papers  (2021-11-29T01:27:42Z)
• Improving Multi-Party Dialogue Discourse Parsing via Domain Integration [25.805553277418813]
  Multi-party conversations are implicitly organized by semantic level correlations across the interactive turns. dialogue discourse analysis can be applied to predict the dependency structure and relations between the elementary discourse units. Existing corpora with dialogue discourse annotation are collected from specific domains with limited sample sizes.
  arXiv  Detail & Related papers  (2021-10-09T09:36:22Z)
• Model-Based Domain Generalization [96.84818110323518]
  We propose a novel approach for the domain generalization problem called Model-Based Domain Generalization. Our algorithms beat the current state-of-the-art methods on the very-recently-proposed WILDS benchmark by up to 20 percentage points.
  arXiv  Detail & Related papers  (2021-02-23T00:59:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.