NVIDIA Nemotron 3.5 Lightning now available in Amazon SageMaker JumpStart

NVIDIA Nemotron 3.5 Lightning is designed for the fast, specialized model execution required by high-volume agentic workloads. With NVIDIA Nemotron 3.5 Lightning on Amazon SageMaker JumpStart, you can access an open model designed for high-volume agentic workloads.

With this launch, you can deploy Nemotron 3.5 Lightning from Amazon SageMaker JumpStart without configuring the serving infrastructure yourself. NVIDIA describes Lightning as the fastest open model in its class for powering always-on agents. It delivers up to 4x higher throughput and up to 30% faster task completion on high-volume agentic workloads. At 30B total parameters with only 3B active, it can run on a single supported GPU. Repetitive, specialized steps in agent workflows can therefore run without frontier-scale infrastructure. In this post, we show you how to deploy Nemotron 3.5 Lightning from SageMaker JumpStart.

Overview of NVIDIA Nemotron 3.5 Lightning

Nemotron 3.5 Lightning is a publicly available foundation model distilled from NVIDIA’s frontier Nemotron 3 Ultra and developed with the Nemotron Coalition. It uses a hybrid Mixture-of-Experts (MoE) architecture and is trained specifically for agentic tool use across popular agent harnesses. It is trained on open datasets and released as an open model, so you can customize it, own the resulting weights, and deploy it wherever your agents run.

The following table summarizes the key specifications and performance characteristics of NVIDIA Nemotron 3.5 Lightning.

Not every agent step needs a frontier model

Always-on agents work continuously: they gather context, observe their environment, reason over what they know, and act. Many of these steps can involve model calls, but they do not all require the same level of capability. Planning a multi-stage workflow or orchestrating sub-agents can demand frontier-level reasoning. Classifying an alert, extracting fields from a form, or checking a record against a policy can often be handled by a smaller, specialized model. These tasks can account for a large share of call volume.

In many cases, running all model-backed steps through a single large model can add frontier-model cost and latency to work that a smaller, specialized model can handle. A system-of-models approach can instead route each step to a model suited to the task.

Nemotron 3.5 Lightning is built for the high-volume end of that system. Its MoE architecture activates 3B of 30B parameters per forward pass, helping maintain high throughput across long, multi-turn sessions. DFlash speculative decoding can further reduce per-token latency. The 1M-token context window allows an agent to carry accumulated state across a long-running session without repeated re-grounding.

If NVIDIA NeMo Switchyard is part of your stack, it can route individual workflow steps across your chosen model pool. Lightning can be selected for high-volume specialized steps where its speed and domain-specific accuracy are a good fit.

Accuracy across reasoning and agentic benchmarks

Across the published evaluations, NVFP4 remains close to BF16 on many tasks, as summarized in the following table. NVIDIA reports that the evaluation recipes and commands used to produce these results are published in NeMo Gym. Accuracy results were measured by NVIDIA under a consistent harness and may differ from vendors’ self-reported numbers.

The following table compares the BF16 and NVFP4 variants of Nemotron 3.5 Lightning across key reasoning and agentic benchmarks.

Customizing for domain accuracy

Organizations can post-train the model with NVIDIA NeMo for domain-specific tools, workflows, and policies, then deploy the resulting model in their chosen environment. The SageMaker JumpStart model card for this launch does not expose JumpStart customization.

Enterprise use cases

Lightning is built for the specialized, high-frequency work inside agent workflows:

• Personal agents: Long-running assistants handling email, calendar, projects, and bookings, with the option to run locally for contextual data.
• Financial services: Extracting data from documents, checking policy rules, monitoring risk signals, and preparing structured summaries.
• Cyber security operations: Enriching alerts, classifying incidents, querying logs, validating controls, correlating indicators, and preparing findings for analysts.
• Telecom: Triaging network alarms, optimizing network configurations, and answering billing questions.
• Retail: Enriching product catalogs, resolving inventory and fulfillment exceptions, assisting product discovery, and handling order, return, and loyalty questions.

Getting started with SageMaker JumpStart

You can deploy Nemotron 3.5 Lightning through Amazon SageMaker JumpStart without manually configuring the serving framework.

Prerequisites

Before you begin, make sure you have:

• An AWS account.
• Appropriately scoped permissions for SageMaker JumpStart.
• Sufficient service quota for GPU instances (for example, ml.g6e.12xlarge, ml.p4d.24xlarge, or ml.p5.48xlarge).

Important: Deploying this model creates a SageMaker AI endpoint that incurs charges while running. For details, see Amazon SageMaker AI pricing. Delete your endpoint when finished to avoid ongoing charges.

Deploy using SageMaker Studio

1. Open Amazon SageMaker Studio.
2. In the navigation pane, choose SageMaker JumpStart.
3. Search for Nemotron 3.5 Lightning.

   Figure 1: Searching for Nemotron 3.5 Lightning in SageMaker JumpStart

4. Select the model card. The NVFP4 model ID is huggingface-reasoning-nemotron-3-5-lightning-30b-a3b-nvfp4. For BF16, use huggingface-reasoning-nemotron-3-5-lightning-30b-a3b-bf16.

   Figure 2: The Nemotron 3.5 Lightning model card in SageMaker JumpStart

   Figure 3: Model card details for Nemotron 3.5 Lightning

5. Choose Deploy.
6. Select your instance type (for example, ml.g6e.24xlarge).

   Figure 4: Selecting an instance type for deployment

7. Review the deployment settings. The defaults are sufficient for most use cases.
8. Choose Deploy to create the endpoint.
9. Wait for the endpoint status to show InService before running inference.

   Figure 5: Endpoint status showing InService

Deploy from Hugging Face

You can also deploy NVIDIA Nemotron 3.5 Lightning to Amazon SageMaker AI from its Hugging Face model page. On the Hugging Face model page, choose Deploy, select Amazon SageMaker AI, and then choose Deploy on SageMaker AI. This opens the SageMaker AI deployment workflow, where you can configure and deploy the model.

Figure 6: Deploying from the Hugging Face model page

Deploy using the SageMaker Python SDK

With SageMaker JumpStart, you can access the NVFP4 and BF16 variants. The following example uses the NVFP4 model ID huggingface-reasoning-nemotron-3-5-lightning-30b-a3b-nvfp4. For BF16, use huggingface-reasoning-nemotron-3-5-lightning-30b-a3b-bf16.

from sagemaker.jumpstart.model import JumpStartModel

model_id = "huggingface-reasoning-nemotron-3-5-lightning-30b-a3b-nvfp4"
model_version = "*"

model = JumpStartModel(model_id=model_id, model_version=model_version)
predictor = model.deploy()

Clean up

To avoid unnecessary charges, delete the SageMaker AI endpoint when you are done:

predictor.delete_endpoint()

Conclusion

NVIDIA Nemotron 3.5 Lightning brings fast, specialized agent execution to Amazon SageMaker JumpStart. NVIDIA reports up to 4x higher throughput and up to 30% faster task completion for high-volume specialized work. Its hybrid MoE architecture, 3B active parameters, DFlash speculative decoding, and 1M-token context are designed for high-volume agent workflows. Because the model is open and designed for customization, you can post-train it for your own tools and policies and retain control of the resulting model.

Nemotron 3.5 Lightning can support specialized agent workloads across personal assistants, financial services, security operations, telecom, and retail. You can deploy it today from SageMaker JumpStart.

Get started by searching for Nemotron 3.5 Lightning in Amazon SageMaker JumpStart. For deployment guidance, see JumpStart foundation model usage in the Amazon SageMaker AI Developer Guide.