Real-TurnTurk: A Multimodal Turkish Corpus for Turn-Taking Prediction

• URL: http://arxiv.org/abs/2608.22071v1
• Date: Sat, 22 Aug 2026 18:29:52 GMT
• Title: Real-TurnTurk: A Multimodal Turkish Corpus for Turn-Taking Prediction
• Abstract summary: Turn-taking is a basic organizational feature of human conversation and remains difficult to model in natural, synchronous dialog systems.<n>This study introduces a multimodal Turkish dataset of unscripted dyadic interactions, comprising synchronized front-facing video, per-speaker audio channels, and time-aligned transcriptions.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Turn-taking is a basic organizational feature of human conversation and remains difficult to model in natural, synchronous dialog systems. While existing research has explored multimodal approaches and large language models for turn-ending prediction, there is a lack of naturalistic conversational corpora specifically addressing turn-taking dynamics in Turkish. This study introduces a multimodal Turkish conversational dataset of unscripted dyadic interactions, comprising synchronized front-facing video, per-speaker audio channels that allow overlapping speech to be attributed to individual speakers, and time-aligned transcriptions. Turn-taking prediction is formulated as a binary classification problem, and a Genetic Algorithm (GA) is employed to optimize interpretable decision rules derived from visual, acoustic, and linguistic features. A hybrid AND-OR rule representation is adopted in the proposed framework to represent the alternative cue combinations that precede a turn transition.

Related papers

• Multimodal Voice Activity Projection for Turn-Taking in Social Robots with Voice-Activity-Related Pretrained Encoders [5.677986854981345]
  Turn-taking prediction is a key requirement for social robots involved in human-human interaction.<n>This work presents a Multimodal Voice Activity Projection framework that extends the audio-only VAP.<n>The proposed approach builds on pretrained audio-visual backbones originally optimized for speech-related tasks.
  arXiv  Detail & Related papers  (2026-07-08T11:34:00Z)
• Turing Patterns for Multimedia: Reaction-Diffusion Multi-Modal Fusion for Language-Guided Video Moment Retrieval [59.55233913340214]
  Video-language models often struggle to capture dynamic, non-linear interactions between temporal video sequences and textual semantics.<n>We propose textbfReaction-Diffusion Multimodal Fusion (RDMF), a novel framework that reimagines video-language alignment as a reaction-diffusion process.<n>Preliminary experiments demonstrate its potential to outperform existing methods in identifying salient video moments.
  arXiv  Detail & Related papers  (2026-06-01T03:02:35Z)
• Syn-TurnTurk: A Synthetic Dataset for Turn-Taking Prediction in Turkish Dialogues [0.0]
  This paper introduces Syn-TurnTurk, a synthetic Turkish dialogue dataset generated using various Qwen Large Language Models (LLMs)<n>We evaluated the dataset using several traditional and deep learning architectures.<n>The results show that advanced models, particularly BI-LSTM and Ensemble (LR+RF) methods, achieve high accuracy (0.839) and AUC scores (0.910)
  arXiv  Detail & Related papers  (2026-04-15T08:39:26Z)
• MOSS-TTSD: Text to Spoken Dialogue Generation [62.04179716555789]
  We present MOSS-TTSD, a spoken dialogue synthesis model for expressive, multi-party conversational speech across multiple languages.<n>With enhanced long-context modeling, MOSS-TTSD generates long-form spoken conversations from dialogue scripts with explicit speaker tags.<n>The model supports various mainstream languages, including English and Chinese, and is adapted to several long-form scenarios.
  arXiv  Detail & Related papers  (2026-03-20T08:23:31Z)
• F-Actor: Controllable Conversational Behaviour in Full-Duplex Models [70.48189107402145]
  We present first open, instruction-following full-stage conversational speech model that can be trained efficiently under typical academic resource constraints.<n>Our model requires just 2,000 hours of data, without relying on large-scale pretraining or multi-stage pretraining.<n>Both the model and training code will be released to enable reproducible research on controllable full-like controllable full-stage speech systems.
  arXiv  Detail & Related papers  (2026-01-16T14:25:57Z)
• Aligning Spoken Dialogue Models from User Interactions [55.192134724622235]
  We propose a novel preference alignment framework to improve spoken dialogue models on realtime conversations from user interactions.<n>We create a dataset of more than 150,000 preference pairs from raw multi-turn speech conversations annotated with AI feedback.<n>Our findings shed light on the importance of a well-calibrated balance among various dynamics, crucial for natural real-time speech dialogue systems.
  arXiv  Detail & Related papers  (2025-06-26T16:45:20Z)
• Prompt-Guided Turn-Taking Prediction [20.002659517035557]
  Turn-taking prediction models are essential components in spoken dialogue systems and conversational robots.<n>Recent approaches leverage transformer-based architectures to predict speech activity continuously and in real-time.<n>We propose a novel model that enables turn-taking prediction to be dynamically controlled via textual prompts.
  arXiv  Detail & Related papers  (2025-06-26T12:49:07Z)
• TransVIP: Speech to Speech Translation System with Voice and Isochrony Preservation [97.54885207518946]
  We introduce a novel model framework TransVIP that leverages diverse datasets in a cascade fashion. We propose two separated encoders to preserve the speaker's voice characteristics and isochrony from the source speech during the translation process. Our experiments on the French-English language pair demonstrate that our model outperforms the current state-of-the-art speech-to-speech translation model.
  arXiv  Detail & Related papers  (2024-05-28T04:11:37Z)
• Cross-Modal Mutual Learning for Cued Speech Recognition [10.225972737967249]
  We propose a transformer based cross-modal mutual learning framework to prompt multi-modal interaction. Our model forces modality-specific information of different modalities to pass through a modality-invariant codebook. We establish a novel large-scale multi-speaker CS dataset for Mandarin Chinese.
  arXiv  Detail & Related papers  (2022-12-02T10:45:33Z)
• Learning to Listen: Modeling Non-Deterministic Dyadic Facial Motion [89.01668641930206]
  We present a framework for modeling interactional communication in dyadic conversations. We autoregressively output multiple possibilities of corresponding listener motion. Our method organically captures the multimodal and non-deterministic nature of nonverbal dyadic interactions.
  arXiv  Detail & Related papers  (2022-04-18T17:58:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.