SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecture

• URL: http://arxiv.org/abs/2605.12500v1
• Date: Tue, 12 May 2026 17:59:58 GMT
• Title: SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecture
• Abstract summary: We introduce SenseNova-U1, a native unified multimodal paradigm built upon NEO-unify.<n>We show two native unified variants, SenseNova-U1-8B-MoT and SenseNova-U1-A3B-MoT, built on dense (8B) and mixture-of-experts (30B-A3B) understanding baselines.<n>We also show detailed model design, data preprocessing, pre-/post-training, and inference strategies to support community research.
• Score: 110.20462227888915
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent large vision-language models (VLMs) remain fundamentally constrained by a persistent dichotomy: understanding and generation are treated as distinct problems, leading to fragmented architectures, cascaded pipelines, and misaligned representation spaces. We argue that this divide is not merely an engineering artifact, but a structural limitation that hinders the emergence of native multimodal intelligence. Hence, we introduce SenseNova-U1, a native unified multimodal paradigm built upon NEO-unify, in which understanding and generation evolve as synergistic views of a single underlying process. We launch two native unified variants, SenseNova-U1-8B-MoT and SenseNova-U1-A3B-MoT, built on dense (8B) and mixture-of-experts (30B-A3B) understanding baselines, respectively. Designed from first principles, they rival top-tier understanding-only VLMs across text understanding, vision-language perception, knowledge reasoning, agentic decision-making, and spatial intelligence. Meanwhile, they deliver strong semantic consistency and visual fidelity, excelling in conventional or knowledge-intensive any-to-image (X2I) synthesis, complex text-rich infographic generation, and interleaved vision-language generation, with or without think patterns. Beyond performance, we show detailed model design, data preprocessing, pre-/post-training, and inference strategies to support community research. Last but not least, preliminary evidence demonstrates that our models extend beyond perception and generation, performing strongly in vision-language-action (VLA) and world model (WM) scenarios. This points toward a broader roadmap where models do not translate between modalities, but think and act across them in a native manner. Multimodal AI is no longer about connecting separate systems, but about building a unified one and trusting the necessary capabilities to emerge from within.

Related papers

• From Pixels to Words -- Towards Native One-Vision Models at Scale [122.87026673966544]
  Current vision models (VLMs) typically stitch together separate image encoders and language decoders via multi-stage alignment.<n>We introduce NEO-ov, a native foundation model that learns cross-frame and pixel-word correspondence end-to-end.<n>NEO-ov largely narrows the gap to modular counterparts while excelling at fine-grained visual perception.
  arXiv  Detail & Related papers  (2026-05-27T17:59:55Z)
• Uni-ViGU: Towards Unified Video Generation and Understanding via A Diffusion-Based Video Generator [25.751284227739617]
  Visual generation incurs substantially higher computational costs than understanding, particularly for video.<n>We propose Uni-ViGU, a framework that unifies video generation and understanding by extending a video generator as the foundation.<n>Experiments demonstrate that Uni-ViGU achieves competitive performance on both video generation and understanding.
  arXiv  Detail & Related papers  (2026-04-09T11:41:58Z)
• UniG2U-Bench: Do Unified Models Advance Multimodal Understanding? [50.92401586025528]
  Unified multimodal models have recently demonstrated strong generative capabilities, yet whether and when generation improves understanding remains unclear.<n>We introduce UniG2U-Bench, a comprehensive benchmark categorizing generation-to-understanding (G2U) evaluation into 7 regimes and 30 subtasks.
  arXiv  Detail & Related papers  (2026-03-03T18:36:16Z)
• Forge-and-Quench: Enhancing Image Generation for Higher Fidelity in Unified Multimodal Models [23.529904770014735]
  This paper introduces a novel perspective: leveraging understanding to enhance the fidelity and detail richness of generated images.<n>We propose Forge-and-Quench, a new unified framework that puts this principle into practice.<n>Experiments show that Forge-and-Quench significantly improves image fidelity and detail across multiple models.
  arXiv  Detail & Related papers  (2026-01-08T08:18:44Z)
• From Pixels to Words -- Towards Native Vision-Language Primitives at Scale [77.93798335498703]
  We launch NEO, a novel family of native Vision-Language Models (VLMs) built from first principles.<n>NEO efficiently develops visual perception from scratch while mitigating vision-language conflicts inside a dense and monolithic model.<n>Our code and models are publicly available at: https://github.com/EvolvingLMMs-Lab/NEO.
  arXiv  Detail & Related papers  (2025-10-16T17:59:58Z)
• Unified Multimodal Model as Auto-Encoder [69.38946823657592]
  We introduce a paradigm regarding understanding as the encoder (I2T) that compresses images into text, and generation as the decoder (T2I) that reconstructs images from that text.<n>Our empirical results suggest that understanding can largely enhance generation (verified on GenEval), while generation, in turn, notably strengthens fine-grained visual perception.
  arXiv  Detail & Related papers  (2025-09-11T17:57:59Z)
• ARPA: A Novel Hybrid Model for Advancing Visual Word Disambiguation Using Large Language Models and Transformers [1.6541870997607049]
  We present ARPA, an architecture that fuses the unparalleled contextual understanding of large language models with the advanced feature extraction capabilities of transformers. ARPA's introduction marks a significant milestone in visual word disambiguation, offering a compelling solution. We invite researchers and practitioners to explore the capabilities of our model, envisioning a future where such hybrid models drive unprecedented advancements in artificial intelligence.
  arXiv  Detail & Related papers  (2024-08-12T10:15:13Z)
• WenLan 2.0: Make AI Imagine via a Multimodal Foundation Model [74.4875156387271]
  We develop a novel foundation model pre-trained with huge multimodal (visual and textual) data. We show that state-of-the-art results can be obtained on a wide range of downstream tasks.
  arXiv  Detail & Related papers  (2021-10-27T12:25:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.