Qwen3.5-Omni Technical Report

• URL: http://arxiv.org/abs/2604.15804v2
• Date: Tue, 21 Apr 2026 03:35:14 GMT
• Title: Qwen3.5-Omni Technical Report
• Abstract summary: Qwen3.5- Omni scales to hundreds of billions of parameters and supports a 256k context length.<n>Qwen3.5- Omni results across 215 audio and audio-visual understanding, reasoning, and interaction subtasks and benchmarks.<n>ARIA dynamically aligns text and speech units, significantly enhancing the stability and prosody of conversational speech.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this work, we present Qwen3.5-Omni, the latest advancement in the Qwen-Omni model family. Representing a significant evolution over its predecessor, Qwen3.5-Omni scales to hundreds of billions of parameters and supports a 256k context length. By leveraging a massive dataset comprising heterogeneous text-vision pairs and over 100 million hours of audio-visual content, the model demonstrates robust omni-modality capabilities. Qwen3.5-Omni-plus achieves SOTA results across 215 audio and audio-visual understanding, reasoning, and interaction subtasks and benchmarks, surpassing Gemini-3.1 Pro in key audio tasks and matching it in comprehensive audio-visual understanding. Architecturally, Qwen3.5-Omni employs a Hybrid Attention Mixture-of-Experts (MoE) framework for both Thinker and Talker, enabling efficient long-sequence inference. The model facilitates sophisticated interaction, supporting over 10 hours of audio understanding and 400 seconds of 720P video (at 1 FPS). To address the inherent instability and unnaturalness in streaming speech synthesis, often caused by encoding efficiency discrepancies between text and speech tokenizers, we introduce ARIA. ARIA dynamically aligns text and speech units, significantly enhancing the stability and prosody of conversational speech with minimal latency impact. Furthermore, Qwen3.5-Omni expands linguistic boundaries, supporting multilingual understanding and speech generation across 10 languages with human-like emotional nuance. Finally, Qwen3.5-Omni exhibits superior audio-visual grounding capabilities, generating script-level structured captions with precise temporal synchronization and automated scene segmentation. Remarkably, we observed the emergence of a new capability in omnimodal models: directly performing coding based on audio-visual instructions, which we call Audio-Visual Vibe Coding.

Related papers

• Qwen-Audio-3.0-ASR Technical Report [52.28849889799101]
  We present Qwen-Audio-3.0-ASR, a Mixture-of-Experts (MoE) LLM-based ASR system.<n>Qwen-Audio-3.0-ASR supports transcription across 30 languages and 16 Chinese dialectal varieties spanning eight major dialect regions.<n>We develop a dedicated streaming variant, Qwen-Audio-3.0-ASR-Streaming, for latency-sensitive applications.
  arXiv  Detail & Related papers  (2026-09-07T14:30:56Z)
• Qwen3-TTS Technical Report [64.94647392030824]
  We present the Qwen3-TTS series, a family of advanced multilingual, controllable, robust, and streaming text-to-speech models.<n>Qwen3-TTS supports state-of-the-art 3-second voice cloning and description-based control.<n>Qwen3-TTS adopts a dual-track LM architecture for real-time synthesis, coupled with two speech tokenizers.
  arXiv  Detail & Related papers  (2026-01-22T03:51:43Z)
• Qwen3-Omni Technical Report [105.11829337290249]
  We present Qwen3- Omni, a single multimodal model that maintains state-of-the-art performance across text, image, audio, and video.<n>Qwen3- Omni matches the performance of same-sized single-modal models within the Qwen series and excels particularly on audio tasks.<n>It supports text interaction in 119 languages, speech understanding in 19 languages, and speech generation in 10 languages.
  arXiv  Detail & Related papers  (2025-09-22T13:26:24Z)
• CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-training [70.31925012315064]
  We present CosyVoice 3, an improved model designed for zero-shot multilingual speech synthesis in the wild.<n>Key features of CosyVoice 3 include a novel speech tokenizer to improve prosody naturalness.<n>Data is expanded from ten thousand hours to one million hours, encompassing 9 languages and 18 Chinese dialects.
  arXiv  Detail & Related papers  (2025-05-23T07:55:21Z)
• Qwen2.5-Omni Technical Report [31.033323728153984]
  We present an end-to-end multimodal model designed to perceive diverse modalities, including text, images, audio, and video, while simultaneously generating text and natural speech responses in a streaming manner.<n>Qwen2.5- Omni achieves state-of-the-art performance on multimodal benchmarks like Omni-Bench.
  arXiv  Detail & Related papers  (2025-03-26T04:17:55Z)
• Nexus: An Omni-Perceptive And -Interactive Model for Language, Audio, And Vision [83.0622534215881]
  This work proposes an industry-level omni-modal large language model (LLM) pipeline that integrates auditory, visual, and linguistic modalities.<n>Our pipeline consists of three main components: First, a modular framework enabling flexible configuration of various encoder-LLM-decoder architectures.<n>Second, a lightweight training strategy that pre-trains audio-language alignment on the state-of-the-art vision-language model Qwen2.5-VL.<n>Third, an audio synthesis pipeline that generates high-quality audio-text data from diverse real-world scenarios.
  arXiv  Detail & Related papers  (2025-02-26T17:26:36Z)
• Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Models [98.34889301515412]
  We develop the Qwen-Audio model and address the limitation by scaling up audio-language pre-training to cover over 30 tasks and various audio types. Qwen-Audio achieves impressive performance across diverse benchmark tasks without requiring any task-specific fine-tuning. We further develop Qwen-Audio-Chat, which allows for input from various audios and text inputs, enabling multi-turn dialogues and supporting various audio-central scenarios.
  arXiv  Detail & Related papers  (2023-11-14T05:34:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.