Single-turn chatbots are evolving into long-running agents that can reason, maintain context, use tools, and run efficiently across many turns to complete complex workflows.
However, these multi-agent workflows cause token counts to grow quickly. Agents plan, call tools, invoke sub-agents, receive information, and then pass history, outputs, and reasoning steps back into the model continuously. As tasks run longer, this constant communication increases costs and the risk of goal drift.
Developers can solve this using a system of models: frontier reasoning models for orchestration and complex planning, and efficient models for high-volume execution, validation, and tool calling.
NVIDIA is releasing NVIDIA Nemotron 3 Ultra, an open model built to help long-running agents complete tasks faster while lowering cost. (View Highlight)
Nemotron 3 Ultra for agent orchestration
Nemotron 3 Ultra is a 550B-parameter Mixture-of-Experts model with 55B active parameters, built for frontier reasoning and orchestration in agentic systems.
Within any agent workflow, most calls are routine, but a critical subset demands deeper reasoning. Nemotron 3 Ultra is built to handle these hard calls: sustaining architectural decisions across coding sessions, synthesizing contradictory evidence across hundreds of research sources, or verifying chip designs across thousands of constraints. (View Highlight)
Nemotron 3 Ultra is also fast. It achieves 5x higher throughput compared to other open models in its class, enabling long-running agents to complete tasks faster and more efficiently. (View Highlight)
Nemotron 3 Ultra is also built for efficiency. In experiments on the SWE-bench and Terminal bench 2.0, it completed benchmarks using fewer total tokens and fewer tokens per turn than comparable models. This lowers the cost for agentic tasks by up to 30%. (View Highlight)
Breakthroughs powering Nemotron 3 Ultra
To mitigate the typical efficiency-accuracy tradeoffs for high-capacity reasoning models, the Nemotron models introduce architectural innovations: Post-trained for agent harness
Nemotron Ultra is post-trained to deliver consistent accuracy across top harnesses. The model is trained using the NVIDIA NeMo RL and Gym open libraries with one of the largest suites of long-running, task-solving, tool-using datasets in the world.
Ultra is optimized for agent-led open harnesses, not just single-turn chat, and is designed to work within workflows where agents plan, call tools, read observations, delegate to sub-agents, validate outputs, and recover from errors across many turns. (View Highlight)
Hybrid Mamba transformer
Mamba layers improve sequence efficiency for long-context workloads, while Transformer layers preserve precise recall when agents need to retrieve specific facts from large context windows. (View Highlight)
NVFP4 precision
The same NVFP4 checkpoint runs on NVIDIA Hopper, NVIDIA Blackwell, and Ampere GPUs. Developers can use one checkpoint across all NVIDIA GPU architectures thanks to specialized NVFP4 quantization kernels. NVFP4 also delivers up to 5x higher throughput per GPU at the same interactivity compared to BF16 on Blackwell. LatentMoE
LatentMoE supports more efficient expert routing, enabling the model to handle workflows spanning reasoning, code generation, tool calls, and domain-specific logic. Multi-token prediction
Multi-token prediction (MTP) helps reduce generation time by predicting multiple future tokens in a single forward pass, improving throughput for long outputs and multi-turn workflows. (View Highlight)
Nemotron 3 Ultra adds Multi-Teacher On-Policy Distillation
Multi-Teacher On-Policy Distillation (MOPD) is a training method in which Ultra learns from multiple specialized teacher models while generating its own attempts during training. More than 10 specialized teacher models are trained, each with its own domain-specific training pipeline. Each teacher scores the model in its area of expertise, helping Ultra improve reasoning across domains more efficiently. Figure 4. A visual guide to MOPD and the specific flow used for Nemotron 3 Ultra
During MOPD, the student model generates rollouts across domains and receives dense reward signals from the corresponding teacher models. To maximize efficiency, MOPD runs asynchronously, with student rollout generation, teacher scoring, and student optimization fully pipelined. (View Highlight)
MOPD is also iterative. After producing an MOPD-trained checkpoint, new rounds of teacher training are initialized from the updated student model, and the improvements are merged into the next MOPD stage.
This co-evolution between students and teachers enables continuous capability improvement and progressively stronger specialization across domains. Users can try MOPD recipes through NeMo-RL, the library that trained the Ultra model. (View Highlight)
Training data for stronger agent reasoning
As with all Nemotron open model launches, much of the training data pipeline is released as permissively as possible. For partners in enterprise and sovereign AI development, training data transparency and provenance matter as much as capability. Domain-specific pre-training data
Building on a 10T token pre-training foundation, Nemotron 3 Ultra adds 212B new tokens targeting three high-value domain gaps:
• 4B tokens of synthetic legal data, increasing the proxy LegalBench average from 64.6% to 74.7%
• 35B tokens of synthesized Wiki-based data, boosting proxy SimpleQA from 40.2% to 50.2%
• 173B refreshed GitHub tokens through Sept. 30, 2025 (View Highlight)
Post-training data and RL environments
This launch is also releasing 10M new SFT samples, 1M new RL tasks across multiple domains, and 15 net-new RL environments, bringing the cumulative Nemotron open data totals to 50M SFT samples, 2M RL tasks, and 55 RL environments.
The result is SWEBench Verified scores between 65% and 70.4% across Pi, OpenHands, Hermes, OpenCode, and Mini SWE Agent—consistent performance regardless of which framework you deploy. (View Highlight)
Nemotron 3 Ultra can be fine-tuned using LoRA, SFT, and reinforcement learning using the NVIDIA NeMo libraries. Developers can get started with the following recipes.
Nemotron 3 Ultra Recipes:
• SFT LoRA: NeMo Automodel (H100 Recipe, GB200 Recipe)
• Full SFT: NeMo Megatron Bridge Recipes
• Reinforcement Learning: NeMo RL GRPO recipe, GRPO LoRA recipe, MOPD recipe Deployment
Deploy Nemotron Ultra using Dynamo Recipes, for KV-aware routing, multi-token prediction (MTP), and disaggregated prefill/decode. (View Highlight)
