Twenty tokens per second, same card

Published September 13, 2026

This is the second afternoon I promised you. In the last post, LIGHTS handed me fourteen tokens per second and I found out why. The closing lines promised I will try the same on DARKNESS: same prompt, same models, same settings. I had a suspicion about what the newer machine would change. Suspicion is not a measurement. This post is the measurement.

Two boxes, one card, one afternoon later

I keep two desktops. LIGHTS lives at the “Lab”, DARKNESS lives at the “Factory”, where I am writing this now. Same GPU in both: RTX 5060 Ti, 16 GB. The machines around the card are not the same: LIGHTS is a Ryzen 9 5900X with 64 GB of DDR4, DARKNESS is a Ryzen 9 7900X with 128 GB of DDR5, one CPU generation newer, double the memory, newer memory generation. Same NVIDIA driver on both: 616.56. One difference I disclose up front, because this is a ledger and not a brochure: ollama is 0.34.0 here, LIGHTS ran its afternoon on 0.33.3. The machines came that way; it is the one variable I could not hold still, and I will keep coming back to it. Another one for your list. The Lab testing scripts were created by AI. Still the same in Factory. This is also something I kept: HW is different, I am trying to measure the effect, trying to keep all else the same. Even the measurer.

The rules of the afternoon

The same rules as the LIGHTS afternoon, no changes. That was my explicit instruction to the agent. Apples to apples, maybe green apples to red ones, I am trying my best. One prompt, fixed, asked the same way every time: “Explain what a database index is and when it stops helping. Two short paragraphs.” Every model ran twice, warm, and the numbers came straight from ollama’s API, nanoseconds and all. Models evicted between configs. Before anything ran, the pre-flight: GPU verified idle, ollama holding nothing, the Windows desktop’s own cut at 1,192 MiB, within a hair of LIGHTS’s 1,219. The script is the same one, rebuilt from the kit the first afternoon produced. First runs on this box carry a cold-start discount, the GPU waking from idle and the model store being a slow local disk, so the comparison below uses warm numbers and says so.

The table

Read the llama3.1 rows again. Same model, same card, same prompt, one setting, five times slower on LIGHTS. Four times slower here. The trap traveled with the software, not with the machine.

The trap, again

Every receipt from the first afternoon reproduced here. During the default run, ollama ps reported the identical signature: context 131072, 23 GB, 36 percent CPU and 64 percent GPU. The server log shows the same contradiction: ollama computes that this card wants a 4,096-token default, then launches the runner with the model’s own maximum, 131072, anyway. ollama show says the model advertises 128K. The version is newer; the behavior is not. This is not a bug some update fixed. It is the product’s standing default, and it will do the same on your box until you set the knob yourself.

And the open case from the first afternoon just closed. The leading suspect then was the model advertising tool support. The real answer sits on the ollama app’s settings screen: a Context length slider, and it sits at 128k. That is where the launcher’s 131072 comes from. The slider allows a window up to its position, capped by each model’s own maximum, so mistral’s 32K stays 32K and fits, which is why it never crawled, while llama3.1’s 128K passes straight through and splits. One setting explains every row of both tables on both machines, and the other box runs the same 128k. The knob was on the settings screen the whole time; I had been reading the logs.

The softer crawl

Here is the honest surprise. The same 128K split that cost LIGHTS five sixths of its speed cost DARKNESS about four fifths: 20.0 tokens per second instead of 14.2. Same signature, softer punishment. My first suspect, the newer platform’s faster bus, died by receipt: this card trains at PCIe 4.0 with 8 lanes here, and LIGHTS’s AM4 platform is 4.0-capable too, so the bus is most likely the same. That leaves two candidates I cannot separate in one afternoon: the CPU side of the split, which is a third of the work and is one Zen generation newer here, and the ollama version itself. Two variables differ between the boxes; I will not pretend one afternoon can attribute between them. The test that would: the same ollama version on both boxes, one visit. And this machine-sensitivity is not one row’s fluke: the qwen 32K row splits too, and moves.

The rule, and the controls

The rule from last time still explains the whole table, now with its source: the window is whatever num_ctx says or, if you say nothing, what the app’s Context length slider allows, capped by each model’s own maximum. The only question is whether that window fits the card. The controls agree. gemma2:9b, whose 8K window always fits, ran 62.9 here against 63.5 on LIGHTS, the same number wearing a different box. Its sibling qwen2.5:7b did 85.5 against 82.1. Whether the default is free or catastrophic depends on exactly one question, and the boxes agree on the answer.

One number did not fit that story at first and I glanced at the screen, gears whirling: qwen2.5:14b at 32K ran a fifth faster here, 33.2 against 27.3, twice, while its own 7B brother moved four percent. The receipt that explains it: ollama ps, captured while the model was resident, shows the 32K row splitting too, 15 GB with 7 percent CPU and 93 percent GPU. This config sits at the card’s edge and carries a slice of its cache across the bus, on both machines. So the qwen row belongs with the crawl, not with the fitted rows: every row that splits moves on the newer machine, every fitted row is identical. What moves them, the newer memory and cores or the newer ollama, one afternoon still cannot price. The shape of the table, though, is one story now.

Where the machine does matter

Not speed. Waiting. The first cold load of mistral took 29 seconds here against 2.5 on LIGHTS, qwen 24.6 against 4.3, because the model store sits on a slow local disk. Another difference between boxes, I can’t help with. Storage devices shaped different. Warm runs never touch the disk, so the table above is clean of it. The card decides how fast you go. The disk decides how long it takes to start going. That is a machine difference that matters, and it is not the one the spec sheets argue about.

The measuring, disclosed

Because this is a ledger: the first afternoon was designed by my assistant and run by me. This one was the assistant’s end to end, per our standing arrangement: it rebuilt the script from the kit, flew the pre-flight, ran every row, collected the receipts, and wrote the draft for this article. The byline and the final wording are mine. The numbers belong to the machine, and the mistakes are on whoever claims them, which is why every one of them carries its receipt.

Receipts vs claims

The published benchmark for this card still says 90.17 tokens per second on mistral 7B. LIGHTS said 81 to 83. DARKNESS says 83.4. Three rigs, one card, one rule: their numbers are true for their box, not yours. And the two boxes now agree on the thing that matters: the trap is the default, not the hardware. It will follow you to any machine until you turn the knob.

I had a suspicion that the newer, bigger machine would move these numbers. It moved two: the punishment for the silent default, softer but very real, and the wait at the start. The card never moved. Suspicion, meet measurement. It was lovely to meet you both.