From Monolithic Blending to Agentic Orchestration: Dynamic Response for Conversational Assistants at Scale

• URL: http://arxiv.org/abs/2609.05758v2
• Date: Wed, 09 Sep 2026 01:43:26 GMT
• Title: From Monolithic Blending to Agentic Orchestration: Dynamic Response for Conversational Assistants at Scale
• Abstract summary: We report a production migration of a customer-support assistant at a large accommodation marketplace.<n>DR replaces a single Qwen3-235B-A22B blended responder with a bounded ReAct orchestrator over typed tools.<n>We attribute each effect to its cause and claim as architecture effects only those measured on identical replayed turns.
• Score: 17.274780841826438
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Conversational assistants can blend retrieval, action selection, escalation, and wording in a single model path, or separate those roles. We report a production migration of a customer-support assistant at a large accommodation marketplace (millions of conversations per month, 11 languages, 10-second P90). Dynamic Response (DR) replaces a single Qwen3-235B-A22B blended responder with a bounded ReAct orchestrator over typed tools plus a smaller generator that writes from a backend-validated context contract. Because the migration also changed prompts, alignment, and serving, we attribute each effect to its cause and claim as architecture effects only those measured on identical replayed turns: typed entity selection moves the reservation selector to a precision-first operating point (precision 8.3% to 89.1%, recall 75.2% to 67.3%), and typed action IDs with a membership check remove observed structured-action hallucination (2.14% to 0.0%). A low-ramp A/B test reproduces the replay escalation reductions: hard-escalation responses fall from 5.60% to 3.08% and soft-escalation responses from 9.56% to 2.49%, while production handoff volume holds roughly steady; self-solve is directional (+5.1 points, 95% CI [-2, +12]). Serving optimizations cut orchestrator P90 latency from 3.87s to 2.24s on a GPU footprint reduced by roughly one-third, and self-hosting reduces estimated annual model-serving cost by more than an order of magnitude.

Related papers

• ReDraft, Don't Just Distill: Reference-Driven Revision for Continual VLLM Post-Training [66.72708893922605]
  Post-training of large multimodal models should add new capabilities while preserving those from pre-training.<n>SFT gives explicit target supervision that learns a task from near-zero accuracy, but its off-policy targets move the model far enough to cause forgetting.<n>We introduce ReDraft, which obtains both from the model's own failures.
  arXiv  Detail & Related papers  (2026-09-15T04:59:03Z)
• Instruction Duplication as an Inference-Time Control Primitive [0.003596046562524959]
  Procedural instruction following is a basic requirement for controllable language-model systems.<n>We introduce instruction duplication, a minimal black-box inference-time control that repeats only the procedural instruction.
  arXiv  Detail & Related papers  (2026-09-03T16:05:21Z)
• Annotations as Rollouts: Efficient and Scalable Reinforcement Learning for Video MLLMs [104.11820606942855]
  Video-ORA-9B decodes in 130 ms instead of 4,780 ms without chain-of-thought.<n>OraRL scales with model size and data, surpassing its backbone from 0.8B to 9B and GRPO up to 100k prompts.
  arXiv  Detail & Related papers  (2026-08-20T18:28:19Z)
• Chameleon: An Adaptive AI-Driven Honeypot Architecture Using Threat-Calibrated Particle Swarm Optimization and Semantic Deception Rapidly-Exploring Random Trees [0.0]
  High-cost commercial deception products share a related weakness in that their response engines are not coupled to real-time model-driven feedback.<n>Mechanism is an openly distributed adaptive honeypot platform introduced here to address both shortcomings.
  arXiv  Detail & Related papers  (2026-08-15T20:30:14Z)
• Efficient Visual Pointing for Embodied AI:Agent-Driven Data Synthesis, Cross-Block Attention, and Iterative Correction [55.11480729304395]
  We describe our PointArena 2026 solution, which achieves 77.2% overall accuracy and ranks second on the benchmark.<n>The ap proach targets three failure modes. First, agent-driven syn thesis builds large semantic and anchor-relative candidate pools.<n>Second, a determinis tic steerable-data pipeline creates a verified 10,000-sample main set, plus reserve samples, using masks, templates, and path verification.
  arXiv  Detail & Related papers  (2026-06-29T06:39:03Z)
• Output Vector Editing for Memorization Mitigation in Large Language Models [68.30351930772788]
  Large language models memorize and reproduce sequences from their training data, creating privacy, copyright, and security risks.<n>Existing neuron-level mitigation methods equate editing with zeroing out neuron activations, but the activation only controls whether a neuron engages.<n>We propose output vector editing, a constrained-optimization edit that locates a small set of neurons responsible for a memorized continuation.
  arXiv  Detail & Related papers  (2026-06-17T07:29:18Z)
• Hierarchical Online Prompt Mutation with Dual-Loop Feedback for Guardrailed Evidence Document Generation: A Production-Evaluation Case Study [0.0]
  HOPM is a hierarchical online prompt mutation framework evaluated on a real marketplace dispute-evidence workflow.<n>The paper includes control setup, sample sizes, confidence intervals, paired tests, prompt-token categories, pseudocode, schema, rubric, guardrail taxonomy, and a constructed example.
  arXiv  Detail & Related papers  (2026-05-31T22:17:44Z)
• Human-Inspired Memory Architecture for LLM Agents [0.9507070656654629]
  We present a biologically-grounded memory architecture comprising six cognitive mechanisms.<n>Each mechanism addresses a specific failure mode of naive memory accumulation.<n>At S-tier scale (50 sessions), dedup-based consolidation yields a +13.3 pp improvement in preference recall.
  arXiv  Detail & Related papers  (2026-05-08T22:52:37Z)
• A Delta-Aware Orchestration Framework for Scalable Multi-Agent Edge Computing [0.0]
  The Synergistic Collapse occurs when scaling beyond 100 agents causes superlinear performance degradation that individual optimizations cannot prevent.<n>Prior work has addressed each contributing factor in isolation: exponential action-space growth, computational redundancy among spatially adjacent agents, and task-agnostic hardware scheduling.<n>We present a framework that addresses all three simultaneously through: (1) Differential Neural Caching, which stores intermediate layer activations and computes only the input deltas; (2) Criticality-Based Action Space Pruning, which organizes agents into priority tiers and reduces coordination complexity from O(n2) to O(n log n) with less than 6% optimal
  arXiv  Detail & Related papers  (2026-04-22T02:54:35Z)
• Towards a Science of Scaling Agent Systems [79.64446272302287]
  We formalize a definition for agent evaluation and characterize scaling laws as the interplay between agent quantity, coordination structure, modelic, and task properties.<n>We derive a predictive model using coordination metrics, that cross-validated R2=0, enabling prediction on unseen task domains.<n>We identify three effects: (1) a tool-coordination trade-off: under fixed computational budgets, tool-heavy tasks suffer disproportionately from multi-agent overhead, and (2) a capability saturation: coordination yields diminishing or negative returns once single-agent baselines exceed 45%.
  arXiv  Detail & Related papers  (2025-12-09T06:52:21Z)
• Learning Adaptive Parallel Reasoning with Language Models [70.1745752819628]
  We propose Adaptive Parallel Reasoning (APR), a novel reasoning framework that enables language models to orchestrate both serialized and parallel computations end-to-end.<n> APR generalizes existing reasoning methods by enabling adaptive multi-threaded inference using spawn() and join() operations.<n>A key innovation is our end-to-end reinforcement learning strategy, optimizing both parent and child inference threads to enhance task success rate without requiring predefined reasoning structures.
  arXiv  Detail & Related papers  (2025-04-21T22:29:02Z)
• Scalable Reinforcement Post-Training Beyond Static Human Prompts: Evolving Alignment via Asymmetric Self-Play [52.3079697845254]
  eva is the first method that allows language models to adaptively create training prompts in both offline and online RL post-training.<n>We show eva can create effective RL curricula and is robust across ablations.
  arXiv  Detail & Related papers  (2024-10-31T08:15:32Z)
• Alexa Teacher Model: Pretraining and Distilling Multi-Billion-Parameter Encoders for Natural Language Understanding Systems [63.713297451300086]
  We present results from a large-scale experiment on pretraining encoders with non-embedding parameter counts ranging from 700M to 9.3B. Their subsequent distillation into smaller models ranging from 17M-170M parameters, and their application to the Natural Language Understanding (NLU) component of a virtual assistant system.
  arXiv  Detail & Related papers  (2022-06-15T20:44:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.