One command turns a DGX Spark into a private AI server
dgx-spark-qwen38 is an open-source installer that turns an NVIDIA DGX Spark, or any GB10 machine (ASUS, Dell, HP, Lenovo, MSI, GIGABYTE, Acer), into a private AI server in one command: measured model lanes, OpenAI and Anthropic APIs, and a local cockpit.
No prompt, image or video leaves the box.
curl -fsSL https://raw.githubusercontent.com/hasso5703/dgx-spark-qwen38/main/get.sh | bashPrefer to read it first? Clone the repo and run ./install.sh: that is all the one-liner does.
Nine lanes, one box, one at a time
A recipe starts one model. This repo installs nine, pinned and measured, and lets them take turns in the same 128 GB pool: load one from the cockpit, and it serves while the others wait on disk.
The 128 GB of unified memory, shared by the CPU and the GPU, held by Qwen3.8-27B while it serves. The other lanes wait on disk until you load one.
71.4tok/s
greedy median, one request
- 135 to 148 tok/s in all at 8 streams
- A window of a million tokens
- GSM8K 94.0% on 200 problems
NVFP4 with a DFlash2 drafter, on SGLang.
- Measured on
- an ASUS Ascent GX10 with 128 GB, the project's reference box
- How
- ./bench.sh, the v1.14.0 campaign
- Source
- BENCHMARKS.md
Every lane, measured on an ASUS Ascent GX10 with 128 GB
| Lane | Headline | Also measured | Method |
|---|---|---|---|
| Qwen3.8-27B NVFP4 with a DFlash2 drafter, on SGLang | 71.4 tok/s greedy median, one request | 135 to 148 tok/s in all at 8 streams. A window of a million tokens. GSM8K 94.0% on 200 problems. | BENCHMARKS.md |
| Qwen3.8-Flash-Next 176B NVFP4 with NEXTN speculation, on SGLang | 47.9 tok/s on code, with FLASH_TIER=context | A 262K-token window. GSM8K 97.41% on 1,314 problems. HumanEval 95.73%, MMLU 90.8%. | docs/flash-lane.md |
| Flash FP4-Local on vLLM | 55.4 tok/s greedy median, at its default effort | 106.6 tok/s in all at 8 streams, at the lean effort. GSM8K 97.64% on 1,314 problems. About half the disk of the flash lane. | docs/flash-local-lane.md |
| Nemotron 3.5 Bolt with its drafter, on vLLM | 140 tok/s rewriting code, one request, reasoning off | 201.2 tok/s in all at 16 streams. Reads images and video. The lane a 64 GB Spark serves first. | docs/nemotron-lane.md |
| Nemotron 3.5 Super VL with its drafter, on vLLM | 69.5 tok/s rewriting code, one request, reasoning off | 81.7 tok/s in all at 4 streams. A window of a million tokens. Reads images and video. | docs/nemotron-super-lane.md |
| Ornith 1.5 122B NVFP4 with its MTP head, on vLLM | 42.4 tok/s greedy median, one request | 90.0 tok/s in all at 8 streams on five real jobs. GSM8K 98.8% on 250 problems, on a second GX10. A window of a million tokens. | docs/ornith-lane.md |
| Kolibri-1 FP8, on vLLM | 48.0 tok/s one request | 121.9 tok/s at 4 streams. 165.8 at 8, 202.0 at 16. Aleph Alpha's FP8 weights. | docs/kolibri-lane.md |
| Qwen-Image 2.1 on SGLang | 7.5 s with its Turbo, per 1024×1024 image | 34.3 s at 40 steps without it. The Turbo fetches 32.5 GB at its first switch. On the same box as the text lanes. | docs/image-lane.md |
| MiniMax-H3 on SGLang | 126.2 to 127.9 s with the Turbo adapter, for 4 s of 480P video | 622.8 to 632.5 s at 50 steps without it. Sound made with the picture. From a sentence. | docs/video-lane.md |
Also on the box: speech on the CPU, beside any lane (Qwen3-ASR hears 30 languages, Kokoro speaks seven), and a System One endpoint that answers with calibrated probabilities instead of text.
The GB10 unified-memory trap
On a Spark the hard part comes after the first model. SGLang's memory accounting misses 25 to 40 GB of transient allocations in the shared pool, and a pool pushed to zero freezes the box until a power cycle. Read how the trap works.
What lets nine lanes live on one box
Pinned and proved
Engine images are pinned by digest and checkpoints by revision. The lane that serves proves it serves before the install ends, and an update is the same command.
One pool, lanes that take turns
The cockpit shows what holds the memory and loads a lane with one button, from a laptop or a phone.
Clients, wired or documented
OpenAI- and Anthropic-compatible APIs, opencode wired to the local model, Claude Code through claude-local, and VS Code Copilot, Open WebUI and Cursor in the clients guide.
Measured, not claimed
GSM8K 94.0% on the 27B and 97.41% on the flash lane, a tool-calling score of 91/100 on tool-eval-bench, every number with its method in BENCHMARKS.md.
Works with the tools you already use
Point your agent or your app at the box instead of a cloud API. Same calls, your hardware.
- Claude Code
- opencode
- Hermes Agent
- Open WebUI
- Cursor
- VS Code Copilot
- LiteLLM
- OpenAI SDK
- Anthropic SDK
http://your-spark:30001/v1http://your-spark:30001Built for every GB10 machine
Same chip, same DGX OS, same installer. It reads how much memory the box has, not which brand is on it.
- NVIDIANVIDIA DGX Spark
- ASUSASUS Ascent GX10
- DellDell Pro Precision with GB10
- HPHP ZGX Nano G1n AI Station
- LenovoLenovo ThinkStation PGX
- MSIMSI EdgeXpert
- GIGABYTEGIGABYTE AI TOP ATOM
- AcerAcer Veriton GN100 AI Mini Workstation
Bought a 64 GB Spark? It installs what fits and serves NVIDIA's Nemotron 3.5 Bolt at 110.8 tok/s, measured at a 64 GB budget, with Qwen3.8-27B one Load away.
See what fits on 64 GBDrive it from a laptop or a phone
The cockpit is installed by default. It shows what holds the memory, loads a lane with one button and runs a coding agent in the browser.
Proven before it ships
releases since 2026-08-15, each with its notes
CI steps every release passes on GitHub
real-client checks for v1.40.2: 8 clients on 7 lanes
views of the project's thread on NVIDIA's forum
- Kevin Welter's write-up of running Qwen3.8 on a DGX Spark follows this recipe.
- MiaAI-Lab builds on its published config.
- heterogeneous-gpu-pd-lab takes its measurements as its DGX Spark reference.
- It is listed in awesome-dgx-spark.
Answers for Spark owners
Each guide answers one question owners ask, with the source of every number.
- How to run a local LLM on a DGX SparkOllama, NVIDIA's playbooks, an engine you tune or a one-command installer: how each runs an LLM on a DGX Spark, what it asks of you, and the speeds measured.
- An OpenAI-compatible API on a DGX SparkPoint Open WebUI, Cursor, Continue, Zed or the OpenAI SDK at your DGX Spark: one base URL on port 30001 and an API key. Any model name works.
- Can a DGX Spark run a 1M-token context?A plain install serves Qwen3.8-27B with a 1,010,000-token window on a DGX Spark. The KV pool behind it, the other 1M lanes, prefill times and how to switch.
- Claude Code with a local model on a DGX SparkRun Claude Code on the model your DGX Spark serves: claude-local sets the endpoint, the key and the limits of the lane, and keeps the session off Anthropic.
- Image and video generation on a DGX SparkQwen-Image 2.1 makes a 1024×1024 image in 7.5 s with its Turbo on a DGX Spark, and MiniMax-H3 4 s of video with its sound in about two minutes.
- opencode with a local model on a DGX SparkThe installer sets up opencode for your DGX Spark's model: an oc launcher that lifts the hidden 32,000-token output cap, limits fitted to the pool.
- Speech recognition and text to speech on a DGX SparkQwen3-ASR hears 30 languages and Kokoro speaks seven, on a DGX Spark's CPU beside any model, behind OpenAI-compatible audio routes.
- Update the AI stack of a DGX Spark without breaking itOne command installs and updates the AI stack of a DGX Spark: pinned images and checkpoints, your choices kept, the serving model proved.
- Which inference engine should you run on a DGX Spark?SGLang, vLLM and llama.cpp on one DGX Spark: the drafter and the content matter more than the engine. Code: 28-32 tok/s SGLang with DSpark, 25-26 llama.cpp.
- Why does my DGX Spark freeze when a model loads?SGLang misses 25-40 GB of transient allocations on a DGX Spark, and a pool pushed to zero freezes it. Why it happens, how to see it coming and how to avoid it.
- Why is Ollama slow on my DGX Spark?Owners report Ollama slow on DGX Spark. What they found, what to check, and one same-harness test: 60.7 tok/s on vLLM against 42.1 on Ollama for GPT-OSS-120B.
Questions owners ask
Does anything leave the box?
No prompt, image or video leaves the box. The installer downloads the pinned engine images and model weights, and the cockpit asks GitHub whether a newer release exists. Every request you send is answered on the machine itself.
Which machines does it run on?
It is built for the NVIDIA DGX Spark and every other GB10 machine on stock DGX OS: ASUS Ascent GX10, Dell Pro Precision with GB10 (launched as Dell Pro Max), HP ZGX Nano G1n, Lenovo ThinkStation PGX, MSI EdgeXpert, GIGABYTE AI TOP ATOM and Acer Veriton GN100. It reads how much memory the box has, not its brand, and it is measured on an ASUS Ascent GX10.
Does it work on a 64 GB DGX Spark?
Yes. On a 64 GB box it installs what fits: NVIDIA's Nemotron 3.5 Bolt serves at 110.8 tok/s (greedy median) and Qwen3.8-27B is one Load away at 67.2 tok/s, both measured at a 64 GB budget on the reference box. A lane that needs 128 GB is refused before a byte is downloaded.
Can I use Claude Code, opencode or Open WebUI with it?
Yes. The box serves OpenAI- and Anthropic-compatible APIs. opencode is wired to the local model at install, and Claude Code, VS Code Copilot, Open WebUI and Cursor are set up in the clients guide of the repo.
How do I update it?
Run the same command again. An update pins the new images and weights the same way, and the lane that serves proves it serves before the update ends.
How fast is it?
Qwen3.8-27B answers at 71.4 tok/s (greedy median of ./bench.sh) and 135 to 148 tok/s in all at 8 streams, with a window of a million tokens. Every lane has its numbers and its method in BENCHMARKS.md.
Is it free?
Yes. It is open source under the MIT license.
Is it made by NVIDIA?
No. It is an independent open-source project. It runs on NVIDIA DGX OS and serves models with engines such as SGLang and vLLM.
Your own AI lab, in one box.
curl -fsSL https://raw.githubusercontent.com/hasso5703/dgx-spark-qwen38/main/get.sh | bash