Why is Ollama slow on my DGX Spark?

This project never measured Ollama. Owners who found it slow on a Spark point at three things: a model running on the CPU instead of the GPU, Ollama's Docker image, and dense models, whose every weight is read at each token. In one owner's same-harness test, vLLM led Ollama alone on three of four models (60.7 against 42.1 tok/s on GPT-OSS-120B) and on all four at 8 concurrent requests, where Ollama's total stayed at its single-stream figure.

Has this project measured Ollama?

No. Its benchmarks cover SGLang, vLLM and llama.cpp on one reference box, an ASUS Ascent GX10, and none of them is Ollama. What this page says about Ollama comes from owners' public reports and from Ollama's own published numbers, each linked, and no comparison here goes beyond what its source measured. Ollama is a good tool with a real place on a Spark, and it partnered with NVIDIA for the launch.

What do owners report about Ollama on a DGX Spark?

• The Docker image. In Very poor performance with Ollama on DGX Spark, the poster ran a 20B GPT OSS model in Docker and called the performance terrible, with GPU use around 89%. A second owner saw the same with gpt-oss-20b and 120b and found LM Studio much faster. The accepted reply says the Ollama image is slow, and a later owner reports normal performance after reinstalling Ollama natively instead of in Docker.
• A GPU at 0%. In Models not using Spark GPU?, an owner who set up Ollama and Open WebUI from NVIDIA's playbook saw 0% GPU in nvidia-smi and the DGX Dashboard, 100% CPU in btop, and every model extremely slow. The replies recommended llama.cpp or vLLM, and the thread does not record a cause.
• A silent CPU split. In vLLM vs Ollama on the same model, the author measured Gemma 4 26B-A4B at 52 tok/s on vLLM with NVFP4 and 40 tok/s on Ollama with Q4_K_M. One Ollama run came back at 16 tok/s: ollama ps showed the model 66% on the CPU and 34% on the GPU, and unloading and reloading it brought it back to 40.
• A dense model. In DGX Spark performance, an owner got 9.46 tok/s from qwen3:32b in Ollama and compared it with a gpt-oss:20b run at 43.73. Replies pointed out that qwen3:32b is dense, all 32B parameters active, while gpt-oss:20b is a mixture of experts with only 3.6B active per token.

Why do dense models run slowly on a GB10?

Single-stream decode on this chip is bound by memory bandwidth: the GB10 advertises 273 GB/s, about 225 GB/s is measurable, and each decode step of a dense model reads all its weights. Ollama's own Spark table shows the pattern: qwen3 32B at q4_K_M decodes at 9.411 tok/s, while the mixtures of experts gpt-oss 20B and 120B, in MXFP4, reach 58.27 and 41.14, measured with Ollama v0.12.6 on firmware 580.95.05 in October 2025.

One way past that limit on a dense model is speculative decoding: a drafter proposes several tokens and the model checks them in one step, so one read of the weights yields more than one token. That is how this project's dense Qwen3.8-27B lane reaches a 71.4 tok/s greedy median on its ./bench.sh probes. Compare like with like, too: on the reference box free prose ran slower than code with every engine and drafter tried, about 2x slower with a block drafter.

What should you check first?

1. Where the model runs. ollama ps shows it in its Processor column, and the ai-muninn write-up treats anything short of 100% GPU as a reason to unload and reload the model.
2. How Ollama is installed. Owners in thread 353456 report the Docker image slow, and one reports normal speed after a native install.
3. What kind of model it is. A dense 32B and a mixture of experts with 3.6B active parameters are different workloads on the same box.
4. How many requests you send at once. In thread 379766, Ollama's total at 8 concurrent requests stayed at about its single-stream figure, and the ai-muninn write-up says Ollama queues requests by default unless OLLAMA_NUM_PARALLEL is raised.

How does Ollama compare with vLLM on the same Spark?

One owner ran four models through both on one DGX Spark with one harness: identical prompt, 400 max tokens, warm-up discarded (thread 379766, August 2026). Ollama ran Q4 builds and vLLM 4-bit builds, so the formats differ:

Model (tok/s)	Ollama alone	vLLM alone	Ollama, 8 at once	vLLM, 8 at once
qwen3-coder-next	59.2	74.8	60.4	289.0
Gemma-4-26B-A4B (MTP on vLLM)	49.6	54.9	51.9	303.6
GPT-OSS-120B	42.1	60.7	42.5	116-153
Qwen3-30B-A3B	85.2	77.1	87.5	313.4

Ollama led alone on Qwen3-30B-A3B, and vLLM on the other three. At 8 concurrent requests vLLM led on all four. Without speculative decoding, NVIDIA's NVFP4 build of that Gemma ran at 30.3 tok/s on vLLM, below the community Q4_K_M in Ollama. The author's caveats: one machine, one operator, one prompt set, speed only, and concurrent figures that moved ±30% between runs.

What did this project measure instead?

None of the figures above are this project's. It measured llama.cpp, which also runs GGUF files, against its SGLang lane on the same Qwen3.8-27B with its frozen battery (greedy):

Workload (tok/s)	llama.cpp UD-Q4_K_XL + MTP n=3, tuned	SGLang + DFlash2 (v1.2)
Agentic coding (code, diffs, tool calls)	25.6	32-40
Reasoning (FR)	27.5	41-44
Free prose EN / FR / DE	17.7 / 18.2 / 18.0	22 / 20 / 17

Its lanes on the reference box include Qwen3.8-27B on SGLang at a 71.4 tok/s greedy median and 135-148 tok/s in all at 8 streams, Nemotron 3.5 Bolt on vLLM at 140 tok/s rewriting code and 201.2 tok/s in all at 16 streams, and Kolibri-1 on vLLM at 48.0 tok/s with one stream and 202.0 with 16. Those are other models than the owners', so they show what each lane delivers, not what Ollama would do with the same models.

Should you move off Ollama?

Not for everything. Ollama is simple to start with, and the ai-muninn author still picks it for quick interactive use. For agents, several users or long sessions, the owner tests above and this project's lanes point to vLLM and SGLang, which kept scaling with concurrent requests. This project's installer sets both up on a Spark in one command, with versions pinned. Every number here is a point-in-time measurement on one machine, and your tok/s will vary with the model, the content and the language.

curl -fsSL https://raw.githubusercontent.com/hasso5703/dgx-spark-qwen38/main/get.sh | bash