Your laptop's speed limit
How fast a local model can talk is one division: memory bandwidth over model size.
Every local-model benchmark you've seen was capped by one division.
To write one token, a model reads every weight it has. All of them, once per token. An 8-billion-parameter model reads 8 billion numbers to produce one piece of a word, then does it again for the next.
The arithmetic on each weight is tiny. Moving it from memory to the chip is not. So the speed limit isn't how fast your chip computes. It's how fast it can read.
tokens per second ≤ memory bandwidth ÷ model size in bytesDo the division
My laptop is an M1 Pro from 2021. Apple quotes 200 GB/s of memory bandwidth.
An 8B model quantised to 4 bits uses half a byte per parameter, so it's about 4 GB, a little more in practice.
200 ÷ 4 = 50 tokens a second. That's the ceiling. Real runs land below it, because nothing is perfectly efficient. They don't land above it.
Here's the same division for a few machines, with 4-bit weights. A dash means the model doesn't fit in memory at all.
| Model | Size at 4-bit | M1 Pro, 32 GB | M6, 32 GB | M5 Max, 128 GB | RTX 5090, 32 GB |
|---|---|---|---|---|---|
| 3B | 1.5 GB | 133 tok/s | 113 tok/s | 409 tok/s | 1,195 tok/s |
| 8B | 4 GB | 50 tok/s | 43 tok/s | 154 tok/s | 448 tok/s |
| 14B | 7 GB | 29 tok/s | 24 tok/s | 88 tok/s | 256 tok/s |
| 32B | 16 GB | 13 tok/s | 11 tok/s | 38 tok/s | 112 tok/s |
| 70B | 35 GB | — | — | 18 tok/s | — |
The bandwidths are 200, 170, 614 and 1,792 GB/s. Everything else is division.
Notice the last row. The 5090 is about nine times faster than my laptop, and it can't run the model at all. Speed doesn't help if the model doesn't fit, and on graphics cards, memory is the expensive part.
Newer isn't faster
Look at the M6 column. It's Apple's newest chip, released in September 2026, and it reads memory at 170 GB/s. My five-year-old M1 Pro does 200. For running models, the new chip is slower.
The generation number barely matters here. The tier does. Pro and Max chips have wider memory buses, and that's the whole game: an M5 Pro reads at 307 GB/s, an M5 Max at 614. If you're buying a Mac to run models, pay for memory bandwidth, not for the newest chip name.
What falls out of this
Quantisation is a speed trick, not just a memory trick. Going from 16 bits to 4 makes the model a quarter of the size, so the ceiling goes up four times. That's why everyone runs 4-bit locally.
Your chip is mostly waiting. When you generate text for one person, the compute units sit idle between reads. That's also why a datacenter is cheaper per token than your laptop: it reads the weights once and uses them for hundreds of people at the same time. You're reading them for one.
Mixture-of-experts changes the maths. An MoE model only reads the experts each token is routed to, so its speed is set by the active parameters, not the total. You still need the memory for all of them.
Long context slows you down. Every new token also reads the attention cache for everything before it. At the start of a conversation that's nothing. Thirty thousand tokens in, it isn't.
Reading the prompt is a different problem. The prompt is processed in parallel, so that part is limited by compute, not bandwidth — and compute is the thing a laptop is actually short of. That's why pasting a long document feels slower than getting the answer.
I checked
I ran Qwen3 8B on my M1 Pro through Ollama. At 4 bits it's 5.2 GB, so the division says 200 ÷ 5.2 = 38 tokens a second, at most.
It did 22. Three runs: 22.1, 21.9 and 22.0.
That's 58% of the ceiling. Below it, never above it, exactly where the maths said it would be. I didn't need the benchmark to know roughly what it would say. I just needed a division.