Nvidia has officially launched Nemotron 3 Super, a 120-billion-parameter open hybrid model engineered to slash the compute cost of deploying AI agents at scale.
Key Takeaways
- Nemotron 3 Super uses a Mixture-of-Experts (MoE) architecture, activating only 12.7B parameters per forward pass to reduce inference overhead.
- It achieves up to 7.5x higher throughput than Qwen3.5-122B-A10B on agent workloads in 8k-in/64k-out settings.
- The model is fully open under the Nvidia Nemotron Open Model License, with checkpoints, training data, and RL environments available on Hugging Face.
Architectural Innovations for Agentic Workloads
Nemotron 3 Super is the second model in Nvidia's Nemotron 3 family, following the December 2025 release of Nemotron 3 Nano. The announcement came around March 10, 2026. The architecture directly addresses two major pain points in multi-step AI agent deployment: the added cost of extended reasoning chains and the ballooning token consumption that can multiply up to 15x in multi-agent pipelines.
The model employs a hybrid Mamba-Transformer backbone across 88 layers. Mamba-2 blocks handle long sequences with linear-time efficiency, while Transformer attention layers preserve precise recall. This combination enables native support for context windows up to one million tokens without the memory penalties typical of pure-attention designs.
Nvidia also integrated a LatentMoE routing system that compresses token embeddings into a low-rank space before sending them to 512 experts per layer, activating 22 at a time. The company claims this allows roughly four times more experts at the same inference cost compared to standard MoE approaches, enabling finer task specialization—such as separating Python logic from SQL handling at the expert level.
Multi-Token Prediction layers with two shared-weight heads accelerate chain-of-thought generation and support native speculative decoding. On structured tasks, Nvidia reports up to three times faster generation.
Training Scale and Benchmark Performance
Nemotron 3 Super was pre-trained on 25 trillion tokens across two phases. The first phase used 20 trillion tokens of broad data; the second used five trillion high-quality tokens tuned for benchmark performance. A final extension phase on 51 billion tokens extended native context to one million tokens. Post-training included supervised fine-tuning on approximately seven million samples and reinforcement learning across 21 environments with over 1.2 million rollouts.
In benchmarks, Nemotron 3 Super scored 83.73 on MMLU-Pro, 90.21 on AIME25, and 60.47 on SWE-Bench using OpenHands. On PinchBench, it reached 85.6%, the highest reported score among open models in its class. For long-context evaluation, it scored 91.64 on RULER 1M.
Compared to GPT-OSS-120B, Nemotron 3 Super delivers 2.2x throughput at 8k input and 64k output. Against Qwen3.5-122B-A10B, that figure reaches 7.5x. Nvidia also reports more than 5x throughput and up to 2x accuracy gains over the prior Nemotron Super generation.
Hardware Optimization and Open Ecosystem
Nvidia trained the model end-to-end in its NVFP4 four-bit floating-point format, optimized for Blackwell GPUs. On B200 hardware, Nvidia says inference runs up to 4x faster compared to FP8 on H100 with no reported accuracy loss. Quantized FP8 and NVFP4 checkpoints retain 99.8% or more of full-precision accuracy.
The model also powers the Nvidia AI-Q research agent, which reached the top position on the Deepresearch Bench leaderboard.
Nemotron 3 Super is fully open under the Nvidia Nemotron Open Model License. Checkpoints in BF16, FP8, and NVFP4 formats, along with pre-training data, post-training samples, and reinforcement learning environments, are available on Hugging Face. Inference is supported through Nvidia NIM, build.nvidia.com, Perplexity, Openrouter, Together AI, Google Cloud, AWS, Azure, and Coreweave, with on-premises options via Dell Enterprise Hub and HPE.
Developers can access training recipes, fine-tuning guides, and inference cookbooks through the NeMo platform using vLLM, SGLang, and TensorRT-LLM.