The Private AI Lab
Testing Nemotron's 3.5 Lightning's 1M-Token Context on the DGX Spark
NVIDIA just shipped a model with four times the context window of what's been running my home agent for months. So I swapped it in, broke my own setup within the first hour, and then threw 700,000 words at it to see if it would lie to me. No guest this episode, just me and my home lab. Jarvis, my home agent, has been running on Qwen3.6-35B-A3B since the start, served through vLLM on the DGX Spark's 128GB of unified memory. Then NVIDIA released Nemotron 3.5 Lightning 30B-A3B in NVFP4, and one number stopped me mid-scroll: a validated 1 million token context window, four times what Qwen gives you at 262,000.I don't trust a spec sheet claim until I've broken it myself, so I swapped the model into Jarvis, found a memory reservation bug in NVIDIA's own deployment recipe within the first hour, fixed it, and then buried a fabricated secret code word 90% of the way through a 700,000 word document to see if the model could actually recall it. I also dug into DSpark, the speculative decoding checkpoint built specifically for this hardware, and measured what it actually does for throughput. This episode is the honest version: what's proven on my own hardware, what's still sitting on NVIDIA's spec sheet, and what I still don't know about whether Nemotron makes Jarvis a better agent day to day. Key topics covered: Why Jarvis's setup (Qwen3.6-35B-A3B on vLLM) needed an indirection layer before any of this mattered NVIDIA Nemotron 3.5 Lightning 30B-A3B-NVFP4's validated 1 million token context window, tested against a 700,000-word document The memory reservation bug in NVIDIA's DGX Spark deployment recipe, and the one-line fix DSpark speculative decoding: how it works, and the acceptance rate that made it worth using Nemotron's hybrid Mamba-2, attention, and mixture-of-experts architecture, and why it matters for running 1M tokens on a single box What's proven on real hardware versus what's still just NVIDIA's own benchmark numbers Why Nemotron isn't confirmed as the better agent brain for Jarvis yet Chapters 00:00 Introduction — what a 1M-token context window actually changes 00:37 Welcome back to The Private AI Lab (no guest this episode) 01:19 Recap: Jarvis running on Qwen3.6-35B-A3B via vLLM on the DGX Spark 01:58 NVIDIA Nemotron 3.5 Lightning lands with a validated 1M-token window 02:49 Deployment: why the Jarvis-LLM alias made the model swap painless 03:28 The memory reservation bug NVIDIA's own recipe doesn't warn about 04:53 The test: a 700,000-word document with one hidden code word 05:51 DSpark speculative decoding: free speed, fully verified 06:47 The reasoning quirk that quietly eats your token budget 07:31 Nemotron vs Qwen: what's proven versus what's still a spec sheet claim 08:01 The hybrid architecture behind why 1M tokens fits on a DGX Spark 09:24 NVIDIA's own SWE-bench Verified numbers, and their limits 09:56 Closing thoughts, and what's coming in the next episode Links mentioned in this episode: NVIDIA Nemotron 3.5 Lightning 30B-A3B-NVFP4 on Hugging Face: https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 NVIDIA Nemotron 3.5 Lightning 30B-A3B-NVFP4-DSpark (speculative decoding checkpoint): https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4-DSpark Qwen3.5-35B-A3B on Hugging Face: https://huggingface.co/Qwen/Qwen3.5-35B-A3B NVIDIA DGX Spark: https://www.nvidia.com/en-us/products/workstations/dgx-spark/ vLLM: https://github.com/vllm-project/vllm
