CDConv: A Benchmark for Contradiction Detection in Chinese Conversations

• URL: http://arxiv.org/abs/2210.08511v1
• Date: Sun, 16 Oct 2022 11:37:09 GMT
• Title: CDConv: A Benchmark for Contradiction Detection in Chinese Conversations
• Authors: Chujie Zheng, Jinfeng Zhou, Yinhe Zheng, Libiao Peng, Zhen Guo, Wenquan Wu, Zhengyu Niu, Hua Wu, Minlie Huang
• Abstract summary: We propose a benchmark for Contradiction Detection in Chinese Conversations, namely CDConv. It contains 12K multi-turn conversations annotated with three typical contradiction categories: Intra-sentence Contradiction, Role Confusion, and History Contradiction.
• Score: 74.78715797366395
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Dialogue contradiction is a critical issue in open-domain dialogue systems. The contextualization nature of conversations makes dialogue contradiction detection rather challenging. In this work, we propose a benchmark for Contradiction Detection in Chinese Conversations, namely CDConv. It contains 12K multi-turn conversations annotated with three typical contradiction categories: Intra-sentence Contradiction, Role Confusion, and History Contradiction. To efficiently construct the CDConv conversations, we devise a series of methods for automatic conversation generation, which simulate common user behaviors that trigger chatbots to make contradictions. We conduct careful manual quality screening of the constructed conversations and show that state-of-the-art Chinese chatbots can be easily goaded into making contradictions. Experiments on CDConv show that properly modeling contextual information is critical for dialogue contradiction detection, but there are still unresolved challenges that require future research.

Related papers

• Red Teaming Language Models for Processing Contradictory Dialogues [30.522961490270532]
  Most language models currently available are prone to self-contradiction during dialogues. This study explores a novel contradictory dialogue processing task that aims to detect and modify contradictory statements in a conversation.
  arXiv  Detail & Related papers  (2024-05-16T14:27:32Z)
• Multimodal Contextual Dialogue Breakdown Detection for Conversational AI Models [1.4199474167684119]
  We introduce a Multimodal Contextual Dialogue Breakdown (MultConDB) model. This model significantly outperforms other known best models by achieving an F1 of 69.27.
  arXiv  Detail & Related papers  (2024-04-11T23:09:18Z)
• Multi-turn Dialogue Comprehension from a Topic-aware Perspective [70.37126956655985]
  This paper proposes to model multi-turn dialogues from a topic-aware perspective. We use a dialogue segmentation algorithm to split a dialogue passage into topic-concentrated fragments in an unsupervised way. We also present a novel model, Topic-Aware Dual-Attention Matching (TADAM) Network, which takes topic segments as processing elements.
  arXiv  Detail & Related papers  (2023-09-18T11:03:55Z)
• FCTalker: Fine and Coarse Grained Context Modeling for Expressive Conversational Speech Synthesis [75.74906149219817]
  Conversational Text-to-Speech (TTS) aims to synthesis an utterance with the right linguistic and affective prosody in a conversational context. We propose a novel expressive conversational TTS model, as termed FCTalker, that learn the fine and coarse grained context dependency at the same time during speech generation.
  arXiv  Detail & Related papers  (2022-10-27T12:20:20Z)
• A Benchmark for Understanding and Generating Dialogue between Characters in Stories [75.29466820496913]
  We present the first study to explore whether machines can understand and generate dialogue in stories. We propose two new tasks including Masked Dialogue Generation and Dialogue Speaker Recognition. We show the difficulty of the proposed tasks by testing existing models with automatic and manual evaluation on DialStory.
  arXiv  Detail & Related papers  (2022-09-18T10:19:04Z)
• Structural Modeling for Dialogue Disentanglement [43.352833140317486]
  Multi-party dialogue context Tangled multi-party dialogue context leads to challenges for dialogue reading comprehension. This work designs a novel model to disentangle multi-party history into threads, by taking dialogue structure features into account.
  arXiv  Detail & Related papers  (2021-10-15T11:28:43Z)
• "How Robust r u?": Evaluating Task-Oriented Dialogue Systems on Spoken Conversations [87.95711406978157]
  This work presents a new benchmark on spoken task-oriented conversations. We study multi-domain dialogue state tracking and knowledge-grounded dialogue modeling. Our data set enables speech-based benchmarking of task-oriented dialogue systems.
  arXiv  Detail & Related papers  (2021-09-28T04:51:04Z)
• Continuity of Topic, Interaction, and Query: Learning to Quote in Online Conversations [23.214585012203084]
  This work studies automatic quotation generation in an online conversation. An encoder-decoder neural framework is employed to continue the context with a quotation. Experiment results on two large-scale datasets in English and Chinese.
  arXiv  Detail & Related papers  (2021-06-18T03:38:48Z)
• I like fish, especially dolphins: Addressing Contradictions in Dialogue Modeling [104.09033240889106]
  We introduce the DialoguE COntradiction DEtection task (DECODE) and a new conversational dataset containing both human-human and human-bot contradictory dialogues. We then compare a structured utterance-based approach of using pre-trained Transformer models for contradiction detection with the typical unstructured approach.
  arXiv  Detail & Related papers  (2020-12-24T18:47:49Z)
• Contextual Dialogue Act Classification for Open-Domain Conversational Agents [10.576497782941697]
  Classifying the general intent of the user utterance in a conversation, also known as Dialogue Act (DA), is a key step in Natural Language Understanding (NLU) for conversational agents. We propose CDAC (Contextual Dialogue Act), a simple yet effective deep learning approach for contextual dialogue act classification. We use transfer learning to adapt models trained on human-human conversations to predict dialogue acts in human-machine dialogues.
  arXiv  Detail & Related papers  (2020-05-28T06:48:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.