Get the gist? Using large language models for few-shot decontextualization

• URL: http://arxiv.org/abs/2310.06254v1
• Date: Tue, 10 Oct 2023 02:00:00 GMT
• Title: Get the gist? Using large language models for few-shot decontextualization
• Authors: Benjamin Kane and Lenhart Schubert
• Abstract summary: We propose a few-shot method of decontextualization using a large language model. We show that this method achieves viable performance on multiple domains using only a small set of examples.
• Score: 3.2271521193192734
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In many NLP applications that involve interpreting sentences within a rich context -- for instance, information retrieval systems or dialogue systems -- it is desirable to be able to preserve the sentence in a form that can be readily understood without context, for later reuse -- a process known as ``decontextualization''. While previous work demonstrated that generative Seq2Seq models could effectively perform decontextualization after being fine-tuned on a specific dataset, this approach requires expensive human annotations and may not transfer to other domains. We propose a few-shot method of decontextualization using a large language model, and present preliminary results showing that this method achieves viable performance on multiple domains using only a small set of examples.

Related papers

• Manual Verbalizer Enrichment for Few-Shot Text Classification [1.860409237919611]
  acrshortmave is an approach for verbalizer construction by enrichment of class labels. Our model achieves state-of-the-art results while using significantly fewer resources.
  arXiv  Detail & Related papers  (2024-10-08T16:16:47Z)
• Language Models for Text Classification: Is In-Context Learning Enough? [54.869097980761595]
  Recent foundational language models have shown state-of-the-art performance in many NLP tasks in zero- and few-shot settings. An advantage of these models over more standard approaches is the ability to understand instructions written in natural language (prompts) This makes them suitable for addressing text classification problems for domains with limited amounts of annotated instances.
  arXiv  Detail & Related papers  (2024-03-26T12:47:39Z)
• Evaluating Large Language Models in Semantic Parsing for Conversational Question Answering over Knowledge Graphs [6.869834883252353]
  This paper evaluates the performance of large language models that have not been explicitly pre-trained on this task. Our results demonstrate that large language models are capable of generating graph queries from dialogues.
  arXiv  Detail & Related papers  (2024-01-03T12:28:33Z)
• Generative Context-aware Fine-tuning of Self-supervised Speech Models [54.389711404209415]
  We study the use of generative large language models (LLM) generated context information. We propose an approach to distill the generated information during fine-tuning of self-supervised speech models. We evaluate the proposed approach using the SLUE and Libri-light benchmarks for several downstream tasks: automatic speech recognition, named entity recognition, and sentiment analysis.
  arXiv  Detail & Related papers  (2023-12-15T15:46:02Z)
• Conjunct Resolution in the Face of Verbal Omissions [51.220650412095665]
  We propose a conjunct resolution task that operates directly on the text and makes use of a split-and-rephrase paradigm in order to recover the missing elements in the coordination structure. We curate a large dataset, containing over 10K examples of naturally-occurring verbal omissions with crowd-sourced annotations. We train various neural baselines for this task, and show that while our best method obtains decent performance, it leaves ample space for improvement.
  arXiv  Detail & Related papers  (2023-05-26T08:44:02Z)
• Stabilized In-Context Learning with Pre-trained Language Models for Few Shot Dialogue State Tracking [57.92608483099916]
  Large pre-trained language models (PLMs) have shown impressive unaided performance across many NLP tasks. For more complex tasks such as dialogue state tracking (DST), designing prompts that reliably convey the desired intent is nontrivial. We introduce a saliency model to limit dialogue text length, allowing us to include more exemplars per query.
  arXiv  Detail & Related papers  (2023-02-12T15:05:10Z)
• Compositional Semantic Parsing with Large Language Models [27.627684573915147]
  We identify challenges in more realistic semantic parsing tasks with larger vocabulary. Our best method is based on least-to-most prompting. We expect similar efforts will lead to new results in other tasks and domains.
  arXiv  Detail & Related papers  (2022-09-29T17:58:28Z)
• Utterance Rewriting with Contrastive Learning in Multi-turn Dialogue [22.103162555263143]
  We introduce contrastive learning and multi-task learning to jointly model the problem. Our proposed model achieves state-of-the-art performance on several public datasets.
  arXiv  Detail & Related papers  (2022-03-22T10:13:27Z)
• In-Context Learning for Few-Shot Dialogue State Tracking [55.91832381893181]
  We propose an in-context (IC) learning framework for few-shot dialogue state tracking (DST) A large pre-trained language model (LM) takes a test instance and a few annotated examples as input, and directly decodes the dialogue states without any parameter updates. This makes the LM more flexible and scalable compared to prior few-shot DST work when adapting to new domains and scenarios.
  arXiv  Detail & Related papers  (2022-03-16T11:58:24Z)
• Text Summarization with Latent Queries [60.468323530248945]
  We introduce LaQSum, the first unified text summarization system that learns Latent Queries from documents for abstractive summarization with any existing query forms. Under a deep generative framework, our system jointly optimize a latent query model and a conditional language model, allowing users to plug-and-play queries of any type at test time. Our system robustly outperforms strong comparison systems across summarization benchmarks with different query types, document settings, and target domains.
  arXiv  Detail & Related papers  (2021-05-31T21:14:58Z)
• MICE: Mining Idioms with Contextual Embeddings [0.0]
  MICEatic expressions can be problematic for natural language processing applications. We present an approach that uses contextual embeddings for that purpose. We show that deep neural networks using both embeddings perform much better than existing approaches.
  arXiv  Detail & Related papers  (2020-08-13T08:56:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.