MultiTurnCleanup: A Benchmark for Multi-Turn Spoken Conversational Transcript Cleanup

• URL: http://arxiv.org/abs/2305.12029v2
• Date: Fri, 27 Oct 2023 17:01:50 GMT
• Title: MultiTurnCleanup: A Benchmark for Multi-Turn Spoken Conversational Transcript Cleanup
• Abstract summary: This study proposes an innovative Multi-Turn Cleanup task for spoken conversational transcripts. We design a data labeling schema to collect the high-quality dataset and provide extensive data analysis.
• Score: 13.4369374707379
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Current disfluency detection models focus on individual utterances each from a single speaker. However, numerous discontinuity phenomena in spoken conversational transcripts occur across multiple turns, hampering human readability and the performance of downstream NLP tasks. This study addresses these phenomena by proposing an innovative Multi-Turn Cleanup task for spoken conversational transcripts and collecting a new dataset, MultiTurnCleanup1. We design a data labeling schema to collect the high-quality dataset and provide extensive data analysis. Furthermore, we leverage two modeling approaches for experimental evaluation as benchmarks for future research.

Related papers

• Character Beyond Speech: Leveraging Role-Playing Evaluation in Audio Large Language Models via Reinforcement Learning [57.22705949022221]
  RoleJudge is an evaluation framework that leverages audio large language models to assess the alignment between speech and character.<n>We introduce RoleChat, the first voice role-playing evaluation dataset enriched with chain-of-thought reasoning annotations.
  arXiv  Detail & Related papers  (2026-04-15T12:39:03Z)
• TurnWise: The Gap between Single- and Multi-turn Language Model Capabilities [57.865325277036455]
  We introduce a new benchmark, TurnWiseEval, for multi-turn capabilities that is directly comparable to single-turn chat evaluation.<n>We additionally introduce our synthetic multi-turn data pipeline TurnWiseData which allows the scalable generation of multi-turn training data.<n>Our experiments with Olmo 3 show that training with multi-turn data is vital to achieving strong multi-turn chat performance.
  arXiv  Detail & Related papers  (2026-03-17T16:40:35Z)
• A Cocktail-Party Benchmark: Multi-Modal dataset and Comparative Evaluation Results [62.01871490859886]
  We introduce the task of Multi-Modal Context-Aware Recognition (MCoRec) in the ninth CHiME Challenge.<n>MCoRec captures natural multi-party conversations where the recordings focus on unscripted, casual group chats.<n>The task requires systems to answer the question "Who speaks when, what, and with whom?" by jointly transcribing each speaker's speech and clustering them into their respective conversations from audio-visual recordings.
  arXiv  Detail & Related papers  (2025-10-27T12:36:43Z)
• VOGUE: A Multimodal Dataset for Conversational Recommendation in Fashion [18.017186369021154]
  VOGUE is a novel dataset of 60 humanhuman dialogues in realistic fashion shopping scenarios.<n>Each dialogue is paired with a shared visual catalogue, item metadata, user fashion profiles and histories, and post-conversation ratings from both Seekers and Assistants.<n>Our initial analyses of VOGUE reveal distinctive dynamics of visually grounded dialogue.
  arXiv  Detail & Related papers  (2025-10-24T04:45:29Z)
• ReviewInstruct: A Review-Driven Multi-Turn Conversations Generation Method for Large Language Models [9.660334829409253]
  Existing methods for generating multi-turn dialogue data struggle to ensure both diversity and quality in instructions.<n>We propose Review-Instruct, a novel framework that synthesizes multi-turn conversations through an iterative "Ask-Respond-Review" process.
  arXiv  Detail & Related papers  (2025-05-16T08:59:07Z)
• Conversation Disentanglement with Bi-Level Contrastive Learning [26.707584899718288]
  Existing methods have two main drawbacks. First, they overemphasize pairwise utterance relations but pay inadequate attention to the utterance-to-context relation modeling. We propose a general disentangle model based on bi-level contrastive learning. It brings closer utterances in the same session while encourages each utterance to be near its clustered session prototypes in the representation space.
  arXiv  Detail & Related papers  (2022-10-27T08:41:46Z)
• Analysis of Joint Speech-Text Embeddings for Semantic Matching [3.6423306784901235]
  We study a joint speech-text embedding space trained for semantic matching by minimizing the distance between paired utterance and transcription inputs. We extend our method to incorporate automatic speech recognition through both pretraining and multitask scenarios.
  arXiv  Detail & Related papers  (2022-04-04T04:50:32Z)
• Findings on Conversation Disentanglement [28.874162427052905]
  We build a learning model that learns utterance-to-utterance and utterance-to-thread classification. Experiments on the Ubuntu IRC dataset show that this approach has the potential to outperform the conventional greedy approach.
  arXiv  Detail & Related papers  (2021-12-10T05:54:48Z)
• Leveraging Pretrained Models for Automatic Summarization of Doctor-Patient Conversations [9.184616102949228]
  We show that fluent and adequate summaries can be generated with limited training data by fine-tuning BART. Using a carefully chosen fine-tuning dataset, this method is shown to be effective at handling longer conversations.
  arXiv  Detail & Related papers  (2021-09-24T20:18:59Z)
• ConvoSumm: Conversation Summarization Benchmark and Improved Abstractive Summarization with Argument Mining [61.82562838486632]
  We crowdsource four new datasets on diverse online conversation forms of news comments, discussion forums, community question answering forums, and email threads. We benchmark state-of-the-art models on our datasets and analyze characteristics associated with the data.
  arXiv  Detail & Related papers  (2021-06-01T22:17:13Z)
• Re-framing Incremental Deep Language Models for Dialogue Processing with Multi-task Learning [14.239355474794142]
  We present a multi-task learning framework to enable the training of one universal incremental dialogue processing model. We show that these tasks provide positive inductive biases to each other with the optimal contribution of each one relying on the severity of the noise from the task.
  arXiv  Detail & Related papers  (2020-11-13T04:31:51Z)
• Response Selection for Multi-Party Conversations with Dynamic Topic Tracking [63.15158355071206]
  We frame response selection as a dynamic topic tracking task to match the topic between the response and relevant conversation context. We propose a novel multi-task learning framework that supports efficient encoding through large pretrained models. Experimental results on the DSTC-8 Ubuntu IRC dataset show state-of-the-art results in response selection and topic disentanglement tasks.
  arXiv  Detail & Related papers  (2020-10-15T14:21:38Z)
• Multi-View Sequence-to-Sequence Models with Conversational Structure for Abstractive Dialogue Summarization [72.54873655114844]
  Text summarization is one of the most challenging and interesting problems in NLP. This work proposes a multi-view sequence-to-sequence model by first extracting conversational structures of unstructured daily chats from different views to represent conversations. Experiments on a large-scale dialogue summarization corpus demonstrated that our methods significantly outperformed previous state-of-the-art models via both automatic evaluations and human judgment.
  arXiv  Detail & Related papers  (2020-10-04T20:12:44Z)
• Topic-Aware Multi-turn Dialogue Modeling [91.52820664879432]
  This paper presents a novel solution for multi-turn dialogue modeling, which segments and extracts topic-aware utterances in an unsupervised way. Our topic-aware modeling is implemented by a newly proposed unsupervised topic-aware segmentation algorithm and Topic-Aware Dual-attention Matching (TADAM) Network.
  arXiv  Detail & Related papers  (2020-09-26T08:43:06Z)
• Filling the Gap of Utterance-aware and Speaker-aware Representation for Multi-turn Dialogue [76.88174667929665]
  A multi-turn dialogue is composed of multiple utterances from two or more different speaker roles. In the existing retrieval-based multi-turn dialogue modeling, the pre-trained language models (PrLMs) as encoder represent the dialogues coarsely. We propose a novel model to fill such a gap by modeling the effective utterance-aware and speaker-aware representations entailed in a dialogue history.
  arXiv  Detail & Related papers  (2020-09-14T15:07:19Z)
• Semi-supervised Learning for Multi-speaker Text-to-speech Synthesis Using Discrete Speech Representation [125.59372403631006]
  We propose a semi-supervised learning approach for multi-speaker text-to-speech (TTS) A multi-speaker TTS model can learn from the untranscribed audio via the proposed encoder-decoder framework with discrete speech representation. We found the model can benefit from the proposed semi-supervised learning approach even when part of the unpaired speech data is noisy.
  arXiv  Detail & Related papers  (2020-05-16T15:47:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.