GPU ranking for local LLMs

38 graphics cards and PCs ranked by an AI performance score based on estimated local LLM speed. Click a column header to sort.
Estimates for llama.cpp (Ollama, LM Studio) · updated 2026-10-14

VRAM

Include previous generation (RTX 40/30, RX 7000) Include Macs / mini PCs (unified memory)

Showing 38

GPU / PC	AI score	VRAM	Bandwidth GB/s	8B tok/s	14B tok/s	35B MoE tok/s	32B tok/s
RTX 5090	100	32 GB	1792	100+	100+	100+	63	Check price
RTX 4090 previous gen	87	24 GB	1008	100+	79	100+	36	Check price
RTX 3090 previous gen	84	24 GB	936	100+	73	100+	33	Check price
Radeon RX 7900 XTX previous gen	81	24 GB	960	100+	64	100+	29	Check price
Mac M3 Ultra (256GB) whole PC (unified memory)	77	256 GB	819	99	55	100+	25	Check price
Mac M4 Max (128GB) whole PC (unified memory)	60	128 GB	546	66	37	100+	16	Check price
RTX 5080	58	16 GB	960	100+	75	△31	△4.4	Check price
Radeon RX 7900 XT previous gen	58	20 GB	800	97	54	△53	△7.6	Check price
RTX 5070 Ti	56	16 GB	896	100+	70	△31	△4.3	Check price
RTX 4080 SUPER previous gen	53	16 GB	736	100+	58	△30	△4.3	Check price
RTX 4080 previous gen	52	16 GB	717	100+	56	△30	△4.2	Check price
RTX 4070 Ti SUPER previous gen	50	16 GB	672	95	53	△30	△4.2	Check price
RTX 5070	47	12 GB	672	95	53	△21	△2.9	Check price
Radeon RX 9070 XT	42	16 GB	640	77	43	△29	△4.1	Check price
Radeon RX 9070	42	16 GB	640	77	43	△29	△4.1	Check price
Radeon RX 7800 XT previous gen	41	16 GB	624	75	42	△29	△4.1	Check price
RTX 3080 (10GB) previous gen	38	10 GB	760	100+	△18	△18	△2.5	Check price
RTX 2080 Ti previous gen	38	11 GB	616	87	△31	△19	△2.7	Check price
RTX 4070 Ti previous gen	37	12 GB	504	71	39	△20	△2.9	Check price
RTX 4070 / SUPER previous gen	37	12 GB	504	71	39	△20	△2.9	Check price
RTX 5060 Ti 16GB	36	16 GB	448	63	35	△28	△4.0	Check price
Radeon RX 6800 previous gen	35	16 GB	512	62	34	△28	△4.0	Check price
Mac M4 Pro (48GB) whole PC (unified memory)	31	48 GB	273	33	18	54	8.2	Check price
Intel Arc B580	30	12 GB	456	55	31	△20	△2.8	Check price
Ryzen AI Max+ 395 (128GB) whole PC (unified memory)	29	128 GB	256	31	17	51	7.7	Check price
RTX 3060 12GB previous gen	28	12 GB	360	51	28	△20	△2.8	Check price
Radeon RX 6700 XT previous gen	26	12 GB	384	46	26	△19	△2.8	Check price
RTX 5060 Ti 8GB	25	8 GB	448	63	△8.6	△16	△2.2	Check price
RTX 5060	25	8 GB	448	63	△8.6	△16	△2.2	Check price
RTX 4060 Ti 16GB previous gen	25	16 GB	288	41	23	△25	△3.7	Check price
RTX 3070 previous gen	25	8 GB	448	63	△8.6	△16	△2.2	Check price
RTX 3060 Ti previous gen	25	8 GB	448	63	△8.6	△16	△2.2	Check price
Radeon RX 9060 XT 16GB	24	16 GB	320	39	21	△25	△3.6	Check price
Radeon RX 7600 XT previous gen	23	16 GB	288	35	19	△24	△3.5	Check price
RTX 5050	19	8 GB	320	45	△8.1	△15	△2.2	Check price
RTX 4060 Ti 8GB previous gen	18	8 GB	288	41	△7.9	△15	△2.2	Check price
RTX 4060 previous gen	17	8 GB	272	38	△7.8	△15	△2.2	Check price
Radeon RX 7600 previous gen	16	8 GB	288	35	△7.7	△15	△2.1	Check price

15 tok/s is about reading speed; above 40 tok/s feels fast. △ = partly offloaded to system RAM; — = won't run.

How to choose

VRAM first: a model that doesn't fit becomes very slow or won't run. 16 GB is the practical starting point for 14B+ models.
Then bandwidth: with the same VRAM, generation speed scales with memory bandwidth — the RTX 5070 Ti is nearly twice as fast as the 5060 Ti 16GB.
Huge models → unified memory: for 70B+ models, Macs or Ryzen AI Max+ PCs with lots of shared memory are the realistic option.

Check whether a specific model fits →

FAQ

How is the AI performance score calculated?

We add up the estimated generation speed (tokens/s) of four reference models — Qwen3 8B, Qwen3 14B, Qwen3.6 35B-A3B (MoE) and Qwen3 32B, all at Q4_K_M with 8K context and 32 GB system RAM — capping each at 100 tok/s, then scale so the top device is 100. The cap means VRAM capacity (whether bigger models fit) matters, not just raw speed.

What does the △ mean?

The model doesn't fit in VRAM and part of it is offloaded to system RAM. It runs, but much more slowly.

Why no prices?

Prices vary a lot by country and change daily. Use the links to check current prices where you live.

Estimates from memory bandwidth for llama.cpp with GGUF Q4_K_M at 8K context; real numbers can differ by ±10–20%.