Related papers: Data Selection for Multi-turn Dialogue Instruction Tuning

Data Selection for Multi-turn Dialogue Instruction Tuning

URL: http://arxiv.org/abs/2604.07892v2
Date: Sun, 12 Apr 2026 02:03:46 GMT
Title: Data Selection for Multi-turn Dialogue Instruction Tuning
Abstract summary: textbfMDS (Multi-turn Dialogue Selection) is a dialogue-level framework that scores whole conversations rather than isolated turns.<n>MDS outperforms strong single-turn selectors, dialogue-level scorers, and baselines on three multi-turn benchmarks and an in-domain Banking test set.
Score: 36.29487117560294
License: http://creativecommons.org/licenses/by/4.0/
Abstract: Instruction-tuned language models increasingly rely on large multi-turn dialogue corpora, but these datasets are often noisy and structurally inconsistent, with topic drift, repetitive chitchat, and mismatched answer formats across turns. We address this from a data selection perspective and propose \textbf{MDS} (Multi-turn Dialogue Selection), a dialogue-level framework that scores whole conversations rather than isolated turns. MDS combines a global coverage stage that performs bin-wise selection in the user-query trajectory space to retain representative yet non-redundant dialogues, with a local structural stage that evaluates within-dialogue reliability through entity-grounded topic grounding and information progress, together with query-answer form consistency for functional alignment. MDS outperforms strong single-turn selectors, dialogue-level LLM scorers, and heuristic baselines on three multi-turn benchmarks and an in-domain Banking test set, achieving the best overall rank across reference-free and reference-based metrics, and is more robust on long conversations under the same training budget. Code and resources are included in the supplementary materials.

Related papers

Beyond Overlap Metrics: Rewarding Reasoning and Preferences for Faithful Multi-Role Dialogue Summarization [5.909152930998754]
Multi-role dialogue summarization requires modeling complex interactions among multiple speakers.<n>Most existing methods optimize for automatic metrics such as ROUGE and BERTScore.<n>We propose a novel framework that couples explicit cognitive-style reasoning with reward-based optimization.
arXiv Detail & Related papers (2026-04-19T01:27:41Z)
Dynamic Knowledge Fusion for Multi-Domain Dialogue State Tracking [14.34658776558061]
We develop a dynamic knowledge fusion framework applicable to multi-domain DST.<n>Results obtained from multi-domain dialogue benchmarks indicate that our method notably improves dialogue tracking accuracy and generalization.
arXiv Detail & Related papers (2026-03-11T03:29:26Z)
How to Choose How to Choose Your Chatbot: A Massively Multi-System MultiReference Data Set for Dialog Metric Evaluation [7.781647951264202]
We release MMSMR, a Massively Multi-System MultiReference dataset to enable future work on metrics and evaluation for dialog. We create and release an 8-reference dialog dataset by extending single-reference evaluation sets. We then train 1750 systems and evaluate them on our novel test set and the DailyDialog dataset.
arXiv Detail & Related papers (2023-05-23T21:33:43Z)
Cue-CoT: Chain-of-thought Prompting for Responding to In-depth Dialogue Questions with LLMs [59.74002011562726]
We propose a novel linguistic cue-based chain-of-thoughts (textitCue-CoT) to provide a more personalized and engaging response. We build a benchmark with in-depth dialogue questions, consisting of 6 datasets in both Chinese and English. Empirical results demonstrate our proposed textitCue-CoT method outperforms standard prompting methods in terms of both textithelpfulness and textitacceptability on all datasets.
arXiv Detail & Related papers (2023-05-19T16:27:43Z)
Re$^3$Dial: Retrieve, Reorganize and Rescale Dialogue Corpus for Long-Turn Open-Domain Dialogue Pre-training [90.3412708846419]
Most dialogues in existing pre-training corpora contain fewer than three turns of dialogue. We propose the Retrieve, Reorganize and Rescale framework (Re$3$Dial) to automatically construct billion-scale long-turn dialogues. By repeating the above process, Re$3$Dial can yield a coherent long-turn dialogue.
arXiv Detail & Related papers (2023-05-04T07:28:23Z)
FCC: Fusing Conversation History and Candidate Provenance for Contextual Response Ranking in Dialogue Systems [53.89014188309486]
We present a flexible neural framework that can integrate contextual information from multiple channels. We evaluate our model on the MSDialog dataset widely used for evaluating conversational response ranking tasks.
arXiv Detail & Related papers (2023-03-31T23:58:28Z)
DIONYSUS: A Pre-trained Model for Low-Resource Dialogue Summarization [127.714919036388]
DIONYSUS is a pre-trained encoder-decoder model for summarizing dialogues in any new domain. Our experiments show that DIONYSUS outperforms existing methods on six datasets.
arXiv Detail & Related papers (2022-12-20T06:21:21Z)
CGoDial: A Large-Scale Benchmark for Chinese Goal-oriented Dialog Evaluation [75.60156479374416]
CGoDial is a new challenging and comprehensive Chinese benchmark for Goal-oriented Dialog evaluation. It contains 96,763 dialog sessions and 574,949 dialog turns totally, covering three datasets with different knowledge sources. To bridge the gap between academic benchmarks and spoken dialog scenarios, we either collect data from real conversations or add spoken features to existing datasets via crowd-sourcing.
arXiv Detail & Related papers (2022-11-21T16:21:41Z)
UniConv: A Unified Conversational Neural Architecture for Multi-domain Task-oriented Dialogues [101.96097419995556]
"UniConv" is a novel unified neural architecture for end-to-end conversational systems in task-oriented dialogues. We conduct comprehensive experiments in dialogue state tracking, context-to-text, and end-to-end settings on the MultiWOZ2.1 benchmark.
arXiv Detail & Related papers (2020-04-29T16:28:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.