TiCo: Time-Controllable Training for Spoken Dialogue Models

• URL: http://arxiv.org/abs/2603.22267v1
• Date: Mon, 23 Mar 2026 17:51:40 GMT
• Title: TiCo: Time-Controllable Training for Spoken Dialogue Models
• Abstract summary: TiCo is a method for enabling spoken dialogue models to follow time-constrained instructions and generate responses with controllable duration.<n>It requires only a small amount of data and no additional question-answer pairs, relying instead on self-generation and reinforcement learning.<n> Experimental results show that TiCo significantly improves adherence to duration constraints while preserving response quality.
• Score: 82.4349748682451
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose TiCo, a simple post-training method for enabling spoken dialogue models (SDMs) to follow time-constrained instructions and generate responses with controllable duration. This capability is valuable for real-world spoken language systems such as voice assistants and interactive agents, where controlling response duration can improve interaction quality. However, despite their strong ability to generate natural spoken responses, existing models lack time awareness and struggle to follow duration-related instructions (e.g., "Please generate a response lasting about 15 seconds"). Through an empirical evaluation of both open-source and commercial SDMs, we show that they frequently fail to satisfy such time-control requirements. TiCo addresses this limitation by enabling models to estimate elapsed speaking time during generation through Spoken Time Markers (STM) (e.g., <10.6 seconds>). These markers help the model maintain awareness of time and adjust the remaining content to meet the target duration. TiCo is simple and efficient: it requires only a small amount of data and no additional question-answer pairs, relying instead on self-generation and reinforcement learning. Experimental results show that TiCo significantly improves adherence to duration constraints while preserving response quality.

Related papers

• MoshiRAG: Asynchronous Knowledge Retrieval for Full-Duplex Speech Language Models [62.05118198431989]
  Asynchronous full-speech models are distinguished by their full-time interactivity and naturalness of AI pauses.<n>Our framework enables the model to identify knowledge-demand conversational queries and ground responses in external information.<n>Our design supports plug-and-play retrieval methods without retraining and demonstrates strong performance on out-of-toe reasoning tasks.
  arXiv  Detail & Related papers  (2026-04-14T16:17:52Z)
• Memory-T1: Reinforcement Learning for Temporal Reasoning in Multi-session Agents [80.33280979339123]
  We introduce Memory-T1, a framework that learns a time-aware memory selection policy using reinforcement learning (RL)<n>On the Time-Dialog benchmark, Memory-T1 boosts a 7B model to an overall score of 67.0%, establishing a new state-of-the-art performance for open-source models.
  arXiv  Detail & Related papers  (2025-12-23T06:37:29Z)
• Game-Time: Evaluating Temporal Dynamics in Spoken Language Models [93.844257719952]
  We introduce the Game-Time Benchmark framework to assess temporal capabilities.<n>Our evaluation of diverse SLM models reveals a clear performance disparity.<n>The GameTime Benchmark provides a foundation for guiding future research toward more temporally-aware conversational AI.
  arXiv  Detail & Related papers  (2025-09-30T15:23:39Z)
• KnowMT-Bench: Benchmarking Knowledge-Intensive Long-Form Question Answering in Multi-Turn Dialogues [58.305425399644086]
  Multi-Turn Long-Form Question Answering (MT-LFQA) is a key application paradigm of Large Language Models (LLMs) in knowledge-intensive domains.<n>We introduce textbfKnowMT-Bench, the textitfirst-ever benchmark designed to systematically evaluate MT-LFQA for LLMs across knowledge-intensive fields.
  arXiv  Detail & Related papers  (2025-09-26T04:32:29Z)
• From What to Respond to When to Respond: Timely Response Generation for Open-domain Dialogue Agents [26.437011114518917]
  TimelyChat benchmark evaluates the capabilities of language models to predict appropriate time intervals and generate time-conditioned responses.<n>We construct a large-scale training dataset by leveraging unlabeled event knowledge from a temporal commonsense knowledge graph.<n>We then train Timer, a dialogue agent designed to proactively predict time intervals and generate timely responses that align with those intervals.
  arXiv  Detail & Related papers  (2025-06-17T07:56:32Z)
• Learning to Reason Over Time: Timeline Self-Reflection for Improved Temporal Reasoning in Language Models [21.579319926212296]
  Large Language Models (LLMs) have emerged as powerful tools for generating coherent text, understanding context, and performing reasoning tasks.<n>They struggle with temporal reasoning, which requires processing time-related information such as event sequencing, durations, and inter-temporal relationships.<n>We introduce TISER, a novel framework that enhances the temporal reasoning abilities of LLMs through a multi-stage process that combines timeline construction with iterative self-reflection.
  arXiv  Detail & Related papers  (2025-04-07T16:51:45Z)
• Full-Duplex-Bench: A Benchmark to Evaluate Full-duplex Spoken Dialogue Models on Turn-taking Capabilities [93.09944267871163]
  FullDuplexBench is a benchmark that systematically evaluates key interactive behaviors.<n>By releasing our benchmark code we aim to advance spoken dialogue modeling and the development of more natural and engaging SDMs.
  arXiv  Detail & Related papers  (2025-03-06T18:59:16Z)
• MTPChat: A Multimodal Time-Aware Persona Dataset for Conversational Agents [23.98067169669452]
  MTPChat is a time-aware persona dialogue dataset that integrates linguistic, visual, and temporal elements within dialogue and persona memory.<n>We propose two time-sensitive tasks: Temporal Next Response Prediction (TNRP) and Temporal Grounding Memory Prediction (TGMP)<n>We present an innovative framework featuring an adaptive temporal module to effectively integrate multimodal streams and capture temporal dependencies.
  arXiv  Detail & Related papers  (2025-02-09T13:00:53Z)
• Aligner-Guided Training Paradigm: Advancing Text-to-Speech Models with Aligner Guided Duration [13.713209707407712]
  We propose a novel Aligner-Guided Training Paradigm that prioritizes accurate duration labelling by training an aligner before the TTS model.<n>Our experimental results show that aligner-guided duration labelling can achieve up to a 16% improvement in word error rate and significantly enhance phoneme and tone alignment.
  arXiv  Detail & Related papers  (2024-12-11T05:39:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.