Nemotron 3 Nano: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning

• URL: http://arxiv.org/abs/2512.20848v1
• Date: Tue, 23 Dec 2025 23:54:32 GMT
• Title: Nemotron 3 Nano: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning
• Abstract summary: Nemotron 3 Nano 30B-A3B is a Mixture-of-Experts hybrid Mamba-Transformer language model.<n>Nemotron 3 Nano was pretrained on 25 trillion text tokens, including more than 3 trillion new unique tokens over Nemotron 2.
• Score: 223.88531320605486
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present Nemotron 3 Nano 30B-A3B, a Mixture-of-Experts hybrid Mamba-Transformer language model. Nemotron 3 Nano was pretrained on 25 trillion text tokens, including more than 3 trillion new unique tokens over Nemotron 2, followed by supervised fine tuning and large-scale RL on diverse environments. Nemotron 3 Nano achieves better accuracy than our previous generation Nemotron 2 Nano while activating less than half of the parameters per forward pass. It achieves up to 3.3x higher inference throughput than similarly-sized open models like GPT-OSS-20B and Qwen3-30B-A3B-Thinking-2507, while also being more accurate on popular benchmarks. Nemotron 3 Nano demonstrates enhanced agentic, reasoning, and chat abilities and supports context lengths up to 1M tokens. We release both our pretrained Nemotron 3 Nano 30B-A3B Base and post-trained Nemotron 3 Nano 30B-A3B checkpoints on Hugging Face.

Related papers

• Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning [269.10652784788374]
  Nemotron 3 Ultra is a 550 billion total and 55 billion active parameter Mixture-of-Experts Hybrid Mamba-Attention language model.<n>We pre-trained Nemotron 3 Ultra on 20 trillion text tokens, then extended the context length to 1M tokens, and post-trained using Supervised Fine Tuning (SFT), Reinforcement Learning (RL), and Multi-teacher On-Policy Distillation (MOPD)
  arXiv  Detail & Related papers  (2026-06-12T22:56:12Z)
• Nemotron 3 Nano Omni: Efficient and Open Multimodal Intelligence [215.36956014523926]
  Nemotron 3 Nano Omni is the latest model in the Nemotron multimodal series.<n>It is the first to support audio inputs alongside text, images, and video.
  arXiv  Detail & Related papers  (2026-04-27T19:49:21Z)
• Nemotron 3 Super: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning [262.0414794728708]
  We describe the pre-training, post-training, and quantization of Nemotron 3 Super.<n>Nemotron 3 Super is a 120 billion (active 12 billion) parameter hybrid Mamba-Attention Mixture-of-Experts model.
  arXiv  Detail & Related papers  (2026-04-14T07:02:32Z)
• NVIDIA Nemotron 3: Efficient and Open Intelligence [227.47413816066845]
  Nemotron 3 family of models deliver strong agentic, reasoning, and conversational capabilities.<n>Nemotron 3 models are post-trained using multi-environment reinforcement learning enabling reasoning, multi-step tool use, and support granular reasoning budget control.<n>Nemotron 3 family uses a Mixture-of-Experts hybrid Mamba-Transformer architecture to provide best-in-class throughput and context lengths of up to 1M tokens.
  arXiv  Detail & Related papers  (2025-12-24T00:24:05Z)
• NVIDIA Nemotron Nano V2 VL [140.7011776408786]
  Nemotron Nano V2 VL builds on Nemotron Nano V2, a hybrid Mamba-Transformer LLM.<n>We are releasing model checkpoints in BF16, FP8, and FP4 formats.
  arXiv  Detail & Related papers  (2025-11-06T00:10:19Z)
• Jet-Nemotron: Efficient Language Model with Post Neural Architecture Search [42.46046429414803]
  Jet-Nemotron is a new family of hybrid-architecture language models.<n>It matches or exceeds the accuracy of leading full-attention models.
  arXiv  Detail & Related papers  (2025-08-21T17:59:08Z)
• NVIDIA Nemotron Nano 2: An Accurate and Efficient Hybrid Mamba-Transformer Reasoning Model [205.51647932840226]
  Nemotron-Nano-9B-v2 is a hybrid Mamba-Transformer language model designed to increase throughput for reasoning workloads.<n>Nemotron-Nano-9B-v2 builds on the Nemotron-H architecture, in which the majority of the self-attention layers in the common Transformer architecture are replaced with Mamba-2 layers.
  arXiv  Detail & Related papers  (2025-08-20T06:00:57Z)
• Llama-Nemotron: Efficient Reasoning Models [131.54943004304656]
  We introduce the Llama-Nemotron series of models, an open family of heterogeneous reasoning models.<n>The family comes in three sizes -- Nano (8B), Super (49B), and Ultra (253B)
  arXiv  Detail & Related papers  (2025-05-02T01:35:35Z)
• Nemotron-H: A Family of Accurate and Efficient Hybrid Mamba-Transformer Models [192.5620883942846]
  Nemotron-H is a family of 8B and 56B/47B hybrid Mamba-Transformer models.<n>We replace the majority of self-attention layers in the common Transformer model architecture with Mamba layers.<n>Nemotron-H models offer either better or on-par accuracy compared to other similarly-sized open-sourced Transformer models.
  arXiv  Detail & Related papers  (2025-04-04T17:41:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.