Advancing an Interdisciplinary Science of Conversation: Insights from a Large Multimodal Corpus of Human Speech

• URL: http://arxiv.org/abs/2203.00674v1
• Date: Tue, 1 Mar 2022 18:50:33 GMT
• Title: Advancing an Interdisciplinary Science of Conversation: Insights from a Large Multimodal Corpus of Human Speech
• Authors: Andrew Reece, Gus Cooney, Peter Bull, Christine Chung, Bryn Dawson, Casey Fitzpatrick, Tamara Glazer, Dean Knox, Alex Liebscher and Sebastian Marin
• Abstract summary: In this report we advance an interdisciplinary science of conversation, with findings from a large, multimodal corpus of 1,656 recorded conversations in spoken English. This 7+ million word, 850 hour corpus totals over 1TB of audio, video, and transcripts, with moment-to-moment measures of vocal, facial, and semantic expression. We report (5) a comprehensive mixed-method report, based on quantitative analysis and qualitative review of each recording, that showcases how individuals from diverse backgrounds alter their communication patterns and find ways to connect.
• Score: 0.12038936091716987
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: People spend a substantial portion of their lives engaged in conversation, and yet our scientific understanding of conversation is still in its infancy. In this report we advance an interdisciplinary science of conversation, with findings from a large, novel, multimodal corpus of 1,656 recorded conversations in spoken English. This 7+ million word, 850 hour corpus totals over 1TB of audio, video, and transcripts, with moment-to-moment measures of vocal, facial, and semantic expression, along with an extensive survey of speaker post conversation reflections. We leverage the considerable scope of the corpus to (1) extend key findings from the literature, such as the cooperativeness of human turn-taking; (2) define novel algorithmic procedures for the segmentation of speech into conversational turns; (3) apply machine learning insights across various textual, auditory, and visual features to analyze what makes conversations succeed or fail; and (4) explore how conversations are related to well-being across the lifespan. We also report (5) a comprehensive mixed-method report, based on quantitative analysis and qualitative review of each recording, that showcases how individuals from diverse backgrounds alter their communication patterns and find ways to connect. We conclude with a discussion of how this large-scale public dataset may offer new directions for future research, especially across disciplinary boundaries, as scholars from a variety of fields appear increasingly interested in the study of conversation.

Related papers

• Conversation Chronicles: Towards Diverse Temporal and Relational Dynamics in Multi-Session Conversations [9.249662593315541]
  We introduce a new 1M multi-session dialogue dataset, Conversation Chronicles, for implementing a long-term conversation setup. We show that dialogue episodes in Conversation Chronicles reflect those properties while maintaining coherent and consistent interactions. We also propose a dialogue model, called ReBot, which consists of chronological summarization and dialogue generation modules.
  arXiv  Detail & Related papers  (2023-10-20T11:06:21Z)
• Multi-turn Dialogue Comprehension from a Topic-aware Perspective [70.37126956655985]
  This paper proposes to model multi-turn dialogues from a topic-aware perspective. We use a dialogue segmentation algorithm to split a dialogue passage into topic-concentrated fragments in an unsupervised way. We also present a novel model, Topic-Aware Dual-Attention Matching (TADAM) Network, which takes topic segments as processing elements.
  arXiv  Detail & Related papers  (2023-09-18T11:03:55Z)
• NewsDialogues: Towards Proactive News Grounded Conversation [72.10055780635625]
  We propose a novel task, Proactive News Grounded Conversation, in which a dialogue system can proactively lead the conversation based on some key topics of the news. To further develop this novel task, we collect a human-to-human Chinese dialogue dataset tsNewsDialogues, which includes 1K conversations with a total of 14.6K utterances.
  arXiv  Detail & Related papers  (2023-08-12T08:33:42Z)
• End-to-end Spoken Conversational Question Answering: Task, Dataset and Model [92.18621726802726]
  In spoken question answering, the systems are designed to answer questions from contiguous text spans within the related speech transcripts. We propose a new Spoken Conversational Question Answering task (SCQA), aiming at enabling the systems to model complex dialogue flows. Our main objective is to build the system to deal with conversational questions based on the audio recordings, and to explore the plausibility of providing more cues from different modalities with systems in information gathering.
  arXiv  Detail & Related papers  (2022-04-29T17:56:59Z)
• Who says like a style of Vitamin: Towards Syntax-Aware DialogueSummarization using Multi-task Learning [2.251583286448503]
  We focus on the association between utterances from individual speakers and unique syntactic structures. Speakers have unique textual styles that can contain linguistic information, such as voiceprint. We employ multi-task learning of both syntax-aware information and dialogue summarization.
  arXiv  Detail & Related papers  (2021-09-29T05:30:39Z)
• Advances in Multi-turn Dialogue Comprehension: A Survey [51.215629336320305]
  We review the previous methods from the perspective of dialogue modeling. We discuss three typical patterns of dialogue modeling that are widely-used in dialogue comprehension tasks.
  arXiv  Detail & Related papers  (2021-03-04T15:50:17Z)
• MultiTalk: A Highly-Branching Dialog Testbed for Diverse Conversations [39.81965687032923]
  We present the MultiTalk dataset, a corpus of over 320,000 sentences of written conversational dialog. We make multiple contributions to study dialog generation in the highly branching setting. Our culminating task is a challenging theory of mind problem, a controllable generation task.
  arXiv  Detail & Related papers  (2021-02-02T02:29:40Z)
• Multi-View Sequence-to-Sequence Models with Conversational Structure for Abstractive Dialogue Summarization [72.54873655114844]
  Text summarization is one of the most challenging and interesting problems in NLP. This work proposes a multi-view sequence-to-sequence model by first extracting conversational structures of unstructured daily chats from different views to represent conversations. Experiments on a large-scale dialogue summarization corpus demonstrated that our methods significantly outperformed previous state-of-the-art models via both automatic evaluations and human judgment.
  arXiv  Detail & Related papers  (2020-10-04T20:12:44Z)
• KdConv: A Chinese Multi-domain Dialogue Dataset Towards Multi-turn Knowledge-driven Conversation [66.99734491847076]
  We propose a Chinese multi-domain knowledge-driven conversation dataset, KdConv, which grounds the topics in multi-turn conversations to knowledge graphs. Our corpus contains 4.5K conversations from three domains (film, music, and travel), and 86K utterances with an average turn number of 19.0.
  arXiv  Detail & Related papers  (2020-04-08T16:25:39Z)
• Detecting depression in dyadic conversations with multimodal narratives and visualizations [1.4824891788575418]
  In this paper, we develop a system that supports humans to analyze conversations. We demonstrate the ability of our system to take in a wide range of multimodal information and automatically generated a prediction score for the depression state of the individual.
  arXiv  Detail & Related papers  (2020-01-13T10:47:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.