MT-OSC: Path for LLMs that Get Lost in Multi-Turn Conversation

• URL: http://arxiv.org/abs/2604.08782v1
• Date: Thu, 09 Apr 2026 21:39:20 GMT
• Title: MT-OSC: Path for LLMs that Get Lost in Multi-Turn Conversation
• Abstract summary: We introduce MT-OSC, a Sequential Condensation framework that efficiently condenses chat history in the background without disrupting the user experience.<n>We evaluate MT-OSC across 13 state-of-the-art LLMs and diverse multi-turn benchmarks, and it consistently narrows the multi-turn performance gap.<n>Our results establish MT-OSC as a scalable scalable solution for multi-turn chats, enabling richer context within constrained input spaces, reducing latency and operational cost, while balancing performance.
• Score: 37.56638517527325
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Large language models (LLMs) suffer significant performance degradation when user instructions and context are distributed over multiple conversational turns, yet multi-turn (MT) interactions dominate chat interfaces. The routine approach of appending full chat history to prompts rapidly exhausts context windows, leading to increased latency, higher computational costs, and diminishing returns as conversations extend. We introduce MT-OSC, a One-off Sequential Condensation framework that efficiently and automatically condenses chat history in the background without disrupting the user experience. MT-OSC employs a Condenser Agent that uses a few-shot inference-based Condenser and a lightweight Decider to selectively retain essential information, reducing token counts by up to 72% in 10-turn dialogues. Evaluated across 13 state-of-the-art LLMs and diverse multi-turn benchmarks, MT-OSC consistently narrows the multi-turn performance gap - yielding improved or preserved accuracy across datasets while remaining robust to distractors and irrelevant turns. Our results establish MT-OSC as a scalable solution for multi-turn chats, enabling richer context within constrained input spaces, reducing latency and operational cost, while balancing performance.

Related papers

• SOMA: Efficient Multi-turn LLM Serving via Small Language Model [64.50946708641608]
  Large Language Models (LLMs) are increasingly deployed in multi-turn dialogue settings.<n>We propose a framework that exploits the early turns of a session to estimate a local response manifold.<n>We distill the mined cases into localized LoRA fine-tuning so the surrogate runs without prompts at inference.
  arXiv  Detail & Related papers  (2026-05-11T23:07:33Z)
• ChatUMM: Robust Context Tracking for Conversational Interleaved Generation [44.19929499646892]
  Unified multimodal models (UMMs) have achieved remarkable progress yet remain constrained by a single-turn interaction paradigm.<n>We present ChatUMM, a conversational unified model that excels at robust context tracking to sustain interleaved multimodal generation.<n>ChatUMM derives its capabilities from an interleaved multi-turn training strategy that models serialized text-image streams as a continuous conversational flow.
  arXiv  Detail & Related papers  (2026-02-06T07:11:50Z)
• Beyond Prompting: Efficient and Robust Contextual Biasing for Speech LLMs via Logit-Space Integration (LOGIC) [8.474586607625737]
  We introduce LOGIC, an efficient and robust framework that operates directly in the decoding layer.<n>LogIC decouples context injection from input processing, ensuring constant-time complexity.<n>Experiments using the Phi-4-MM model across 11 multilingual locales demonstrate that LOGIC achieves an average 9% relative reduction in Entity WER.
  arXiv  Detail & Related papers  (2026-01-21T19:08:45Z)
• DYCP: Dynamic Context Pruning for Long-Form Dialogue with LLMs [8.451732843394973]
  Large Language Models (LLMs) exhibit increased response latency and degraded answer quality as dialogue length grows.<n>We introduce DyCP, a lightweight context management method that dynamically segment and retrieve relevant memory at query time.
  arXiv  Detail & Related papers  (2026-01-12T20:47:50Z)
• Reactive Transformer (RxT) -- Stateful Real-Time Processing for Event-Driven Reactive Language Models [0.0]
  Reactive Transformer (RxT) is a novel architecture designed to overcome limitations by shifting from a data-driven to an event-driven paradigm.<n> RxT processes each conversational turn as a discrete event in real-time, maintaining context in an integrated Short-Term Memory (STM) system.<n>We show that RxT achieves low latency, enabling truly real-time, stateful, and economically viable long-form conversations.
  arXiv  Detail & Related papers  (2025-10-03T23:18:07Z)
• KnowMT-Bench: Benchmarking Knowledge-Intensive Long-Form Question Answering in Multi-Turn Dialogues [58.305425399644086]
  Multi-Turn Long-Form Question Answering (MT-LFQA) is a key application paradigm of Large Language Models (LLMs) in knowledge-intensive domains.<n>We introduce textbfKnowMT-Bench, the textitfirst-ever benchmark designed to systematically evaluate MT-LFQA for LLMs across knowledge-intensive fields.
  arXiv  Detail & Related papers  (2025-09-26T04:32:29Z)
• LoopServe: An Adaptive Dual-phase LLM Inference Acceleration System for Multi-Turn Dialogues [59.12542274007847]
  LoopServe is an adaptive dual-phase inference framework for large language models in multi-turn dialogues.<n>LoopServe consistently achieves superior effectiveness compared to existing baselines.
  arXiv  Detail & Related papers  (2025-07-18T06:12:08Z)
• Task-Oriented Feature Compression for Multimodal Understanding via Device-Edge Co-Inference [54.53508601749513]
  We propose a task-oriented feature compression (TOFC) method for multimodal understanding in a device-edge co-inference framework.<n>To enhance compression efficiency, multiple entropy models are adaptively selected based on the characteristics of the visual features.<n>Results show that TOFC achieves up to 52% reduction in data transmission overhead and 63% reduction in system latency.
  arXiv  Detail & Related papers  (2025-03-17T08:37:22Z)
• DroidSpeak: KV Cache Sharing for Cross-LLM Communication and Multi-LLM Serving [14.700431530275132]
  We introduce DroidSpeak, the first distributed LLM inference system that enables KV cache reuse across distributed nodes.<n>We show that DroidSpeak achieves up to 4x throughput improvement and about 3.1x faster prefill (time to first token)<n>Experiments on diverse datasets and model pairs demonstrate that DroidSpeak achieves up to 4x throughput improvement and about 3.1x faster prefill.
  arXiv  Detail & Related papers  (2024-11-05T05:41:41Z)
• Beyond the Turn-Based Game: Enabling Real-Time Conversations with Duplex Models [66.24055500785657]
  Traditional turn-based chat systems prevent users from verbally interacting with system while it is generating responses. To overcome these limitations, we adapt existing LLMs to listen users while generating output and provide users with instant feedback. We build a dataset consisting of alternating time slices of queries and responses as well as covering typical feedback types in instantaneous interactions.
  arXiv  Detail & Related papers  (2024-06-22T03:20:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.