Speaker Verification in Agent-Generated Conversations

• URL: http://arxiv.org/abs/2405.10150v2
• Date: Thu, 6 Jun 2024 03:36:16 GMT
• Title: Speaker Verification in Agent-Generated Conversations
• Authors: Yizhe Yang, Palakorn Achananuparp, Heyan Huang, Jing Jiang, Ee-Peng Lim,
• Abstract summary: The recent success of large language models (LLMs) has attracted widespread interest to develop role-playing conversational agents personalized to the characteristics and styles of different speakers to enhance their abilities to perform both general and special purpose dialogue tasks. This study introduces a novel evaluation challenge: speaker verification in agent-generated conversations, which aimed to verify whether two sets of utterances originate from the same speaker.
• Score: 47.6291644653831
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The recent success of large language models (LLMs) has attracted widespread interest to develop role-playing conversational agents personalized to the characteristics and styles of different speakers to enhance their abilities to perform both general and special purpose dialogue tasks. However, the ability to personalize the generated utterances to speakers, whether conducted by human or LLM, has not been well studied. To bridge this gap, our study introduces a novel evaluation challenge: speaker verification in agent-generated conversations, which aimed to verify whether two sets of utterances originate from the same speaker. To this end, we assemble a large dataset collection encompassing thousands of speakers and their utterances. We also develop and evaluate speaker verification models under experiment setups. We further utilize the speaker verification models to evaluate the personalization abilities of LLM-based role-playing models. Comprehensive experiments suggest that the current role-playing models fail in accurately mimicking speakers, primarily due to their inherent linguistic characteristics.

Related papers

• SPECTRUM: Speaker-Enhanced Pre-Training for Long Dialogue Summarization [48.284512017469524]
  Multi-turn dialogues are characterized by their extended length and the presence of turn-taking conversations. Traditional language models often overlook the distinct features of these dialogues by treating them as regular text. We propose a speaker-enhanced pre-training method for long dialogue summarization.
  arXiv  Detail & Related papers  (2024-01-31T04:50:00Z)
• ELF: Encoding Speaker-Specific Latent Speech Feature for Speech Synthesis [5.824018496599849]
  We propose a novel method for modeling numerous speakers. It enables expressing the overall characteristics of speakers in detail like a trained multi-speaker model.
  arXiv  Detail & Related papers  (2023-11-20T13:13:24Z)
• Joining the Conversation: Towards Language Acquisition for Ad Hoc Team Play [1.370633147306388]
  We propose and consider the problem of cooperative language acquisition as a particular form of the ad hoc team play problem. We present a probabilistic model for inferring a speaker's intentions and a listener's semantics from observing communications between a team of language-users.
  arXiv  Detail & Related papers  (2023-05-20T16:59:27Z)
• Enhanced Speaker-aware Multi-party Multi-turn Dialogue Comprehension [43.352833140317486]
  Multi-party multi-turn dialogue comprehension brings unprecedented challenges. Most existing methods deal with dialogue contexts as plain texts. We propose an enhanced speaker-aware model with masking attention and heterogeneous graph networks.
  arXiv  Detail & Related papers  (2021-09-09T07:12:22Z)
• Investigating on Incorporating Pretrained and Learnable Speaker Representations for Multi-Speaker Multi-Style Text-to-Speech [54.75722224061665]
  In this work, we investigate different speaker representations and proposed to integrate pretrained and learnable speaker representations. The FastSpeech 2 model combined with both pretrained and learnable speaker representations shows great generalization ability on few-shot speakers.
  arXiv  Detail & Related papers  (2021-03-06T10:14:33Z)
• Filling the Gap of Utterance-aware and Speaker-aware Representation for Multi-turn Dialogue [76.88174667929665]
  A multi-turn dialogue is composed of multiple utterances from two or more different speaker roles. In the existing retrieval-based multi-turn dialogue modeling, the pre-trained language models (PrLMs) as encoder represent the dialogues coarsely. We propose a novel model to fill such a gap by modeling the effective utterance-aware and speaker-aware representations entailed in a dialogue history.
  arXiv  Detail & Related papers  (2020-09-14T15:07:19Z)
• Active Speakers in Context [88.22935329360618]
  Current methods for active speak er detection focus on modeling short-term audiovisual information from a single speaker. This paper introduces the Active Speaker Context, a novel representation that models relationships between multiple speakers over long time horizons. Our experiments show that a structured feature ensemble already benefits the active speaker detection performance.
  arXiv  Detail & Related papers  (2020-05-20T01:14:23Z)
• Speech Enhancement using Self-Adaptation and Multi-Head Self-Attention [70.82604384963679]
  This paper investigates a self-adaptation method for speech enhancement using auxiliary speaker-aware features. We extract a speaker representation used for adaptation directly from the test utterance.
  arXiv  Detail & Related papers  (2020-02-14T05:05:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.