Investigating Low-Cost LLM Annotation for~Spoken Dialogue Understanding Datasets

• URL: http://arxiv.org/abs/2406.13269v1
• Date: Wed, 19 Jun 2024 06:59:57 GMT
• Title: Investigating Low-Cost LLM Annotation for~Spoken Dialogue Understanding Datasets
• Authors: Lucas Druart, Valentin Vielzeuf, Yannick Estève,
• Abstract summary: In spoken Task-Oriented Dialogue (TOD) systems, the choice of the semantic representation describing the users' requests is key to a smooth interaction. This paper provides insights into automatic enhancement of spoken dialogue datasets' semantic representations.
• Score: 9.78470355087662
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In spoken Task-Oriented Dialogue (TOD) systems, the choice of the semantic representation describing the users' requests is key to a smooth interaction. Indeed, the system uses this representation to reason over a database and its domain knowledge to choose its next action. The dialogue course thus depends on the information provided by this semantic representation. While textual datasets provide fine-grained semantic representations, spoken dialogue datasets fall behind. This paper provides insights into automatic enhancement of spoken dialogue datasets' semantic representations. Our contributions are three fold: (1) assess the relevance of Large Language Model fine-tuning, (2) evaluate the knowledge captured by the produced annotations and (3) highlight semi-automatic annotation implications.

Related papers

• Learning from Emotions, Demographic Information and Implicit User Feedback in Task-Oriented Document-Grounded Dialogues [59.516187851808375]
  We introduce FEDI, the first English dialogue dataset for task-oriented document-grounded dialogues annotated with demographic information, user emotions and implicit feedback. Our experiments with FLAN-T5, GPT-2 and LLaMA-2 show that these data have the potential to improve task completion and the factual consistency of the generated responses and user acceptance.
  arXiv  Detail & Related papers  (2024-01-17T14:52:26Z)
• InstructERC: Reforming Emotion Recognition in Conversation with Multi-task Retrieval-Augmented Large Language Models [9.611864685207056]
  We propose a novel approach, InstructERC, to reformulate the emotion recognition task from a discriminative framework to a generative framework based on Large Language Models (LLMs) InstructERC makes three significant contributions: (1) it introduces a simple yet effective retrieval template module, which helps the model explicitly integrate multi-granularity dialogue supervision information; (2) we introduce two additional emotion alignment tasks, namely speaker identification and emotion prediction tasks, to implicitly model the dialogue role relationships and future emotional tendencies in conversations; and (3) Pioneeringly, we unify emotion labels across benchmarks through the feeling wheel to fit real application scenarios.
  arXiv  Detail & Related papers  (2023-09-21T09:22:07Z)
• Context-Dependent Embedding Utterance Representations for Emotion Recognition in Conversations [1.8126187844654875]
  We approach Emotion Recognition in Conversations leveraging the conversational context. We propose context-dependent embedding representations of each utterance. The effectiveness of our approach is validated on the open-domain DailyDialog dataset and on the task-oriented EmoWOZ dataset.
  arXiv  Detail & Related papers  (2023-04-17T12:37:57Z)
• FCC: Fusing Conversation History and Candidate Provenance for Contextual Response Ranking in Dialogue Systems [53.89014188309486]
  We present a flexible neural framework that can integrate contextual information from multiple channels. We evaluate our model on the MSDialog dataset widely used for evaluating conversational response ranking tasks.
  arXiv  Detail & Related papers  (2023-03-31T23:58:28Z)
• Improve Retrieval-based Dialogue System via Syntax-Informed Attention [46.79601705850277]
  We propose SIA, Syntax-Informed Attention, considering both intra- and inter-sentence syntax information. We evaluate our method on three widely used benchmarks and experimental results demonstrate the general superiority of our method on dialogue response selection.
  arXiv  Detail & Related papers  (2023-03-12T08:14:16Z)
• Dialogue Term Extraction using Transfer Learning and Topological Data Analysis [0.8185867455104834]
  We explore different features that can enable systems to discover realizations of domains, slots, and values in dialogues in a purely data-driven fashion. To examine the utility of each feature set, we train a seed model based on the widely used MultiWOZ data-set. Our method outperforms the previously proposed approach that relies solely on word embeddings.
  arXiv  Detail & Related papers  (2022-08-22T17:04:04Z)
• Manual-Guided Dialogue for Flexible Conversational Agents [84.46598430403886]
  How to build and use dialogue data efficiently, and how to deploy models in different domains at scale can be critical issues in building a task-oriented dialogue system. We propose a novel manual-guided dialogue scheme, where the agent learns the tasks from both dialogue and manuals. Our proposed scheme reduces the dependence of dialogue models on fine-grained domain ontology, and makes them more flexible to adapt to various domains.
  arXiv  Detail & Related papers  (2022-08-16T08:21:12Z)
• Language Model as an Annotator: Exploring DialoGPT for Dialogue Summarization [29.887562761942114]
  We show how DialoGPT, a pre-trained model for conversational response generation, can be developed as an unsupervised dialogue annotator. We apply DialoGPT to label three types of features on two dialogue summarization datasets, SAMSum and AMI, and employ pre-trained and non pre-trained models as our summarizes.
  arXiv  Detail & Related papers  (2021-05-26T13:50:13Z)
• Dialogue History Matters! Personalized Response Selectionin Multi-turn Retrieval-based Chatbots [62.295373408415365]
  We propose a personalized hybrid matching network (PHMN) for context-response matching. Our contributions are two-fold: 1) our model extracts personalized wording behaviors from user-specific dialogue history as extra matching information. We evaluate our model on two large datasets with user identification, i.e., personalized dialogue Corpus Ubuntu (P- Ubuntu) and personalized Weibo dataset (P-Weibo)
  arXiv  Detail & Related papers  (2021-03-17T09:42:11Z)
• Topic-Aware Multi-turn Dialogue Modeling [91.52820664879432]
  This paper presents a novel solution for multi-turn dialogue modeling, which segments and extracts topic-aware utterances in an unsupervised way. Our topic-aware modeling is implemented by a newly proposed unsupervised topic-aware segmentation algorithm and Topic-Aware Dual-attention Matching (TADAM) Network.
  arXiv  Detail & Related papers  (2020-09-26T08:43:06Z)
• Improving Conversational Recommender Systems via Knowledge Graph based Semantic Fusion [77.21442487537139]
  Conversational recommender systems (CRS) aim to recommend high-quality items to users through interactive conversations. First, the conversation data itself lacks of sufficient contextual information for accurately understanding users' preference. Second, there is a semantic gap between natural language expression and item-level user preference.
  arXiv  Detail & Related papers  (2020-07-08T11:14:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.