Best local LLMs for 32 GB VRAM

32 GB is the top of consumer GPUs: 30B-class models at high-quality quantization. Below are the 29 models we track, with the highest-quality quantization that fits in 32 GB, memory needed and estimated speed (8K context, data updated 2026-09-27).

Calculate with your own settings →

Models that fit in 32 GB (24)

Speed estimated on the RTX 5090 (1792 GB/s).

Model	Best quant	Memory (8K)	Verdict	Speed
Qwen3.5 4B Qwen · Lightweight, image input	BF16	10.5GB	Comfortable	Fast (100+ tok/s)
Gemma 4 E4B Gemma · Lightweight for phones/laptops	BF16	17.2GB	Comfortable	Fast (78 tok/s)
Qwen3 8B Qwen · The go-to 8B	BF16	18.6GB	Comfortable	Fast (77 tok/s)
Llama 3.1 8B Llama · Classic, many fine-tunes	BF16	18.1GB	Comfortable	Fast (78 tok/s)
Ministral 3 8B Mistral	BF16	20.0GB	Comfortable	Fast (70 tok/s)
DeepSeek-R1-0528-Qwen3-8B DeepSeek · Reasoning (thinking) model	BF16	18.6GB	Comfortable	Fast (77 tok/s)
llm-jp-4 8B thinking Japanese · Japanese-focused (Japan)	BF16	19.3GB	Comfortable	Fast (73 tok/s)
Qwen3-Swallow 8B Japanese · Japanese-enhanced (Institute of Science Tokyo)	BF16	18.6GB	Comfortable	Fast (77 tok/s)
Qwen3.5 9B Qwen · Image input	BF16	20.6GB	Comfortable	Fast (65 tok/s)
Gemma 4 12B Gemma · Mid-size, good multilingual	BF16	25.6GB	Comfortable	Fast (52 tok/s)
Ministral 3 14B Mistral	BF16	30.5GB	Tight	Fast (45 tok/s)
Qwen3 14B Qwen	Q8_0	18.0GB	Comfortable	Fast (80 tok/s)
gpt-oss-20b OpenAI · OpenAI open-weight model (MXFP4)	Q4_K_M (MXFP4)	13.2GB	Comfortable	Fast (100+ tok/s)
Devstral Small 2 24B Mistral · Coding	Q8_0	28.0GB	Comfortable	Fast (49 tok/s)
Gemma 4 26B-A4B Gemma · Fast MoE	Q8_0	29.0GB	Tight	Fast (100+ tok/s)
Qwen3.8 27B Qwen · Latest generation (Aug 2026)	Q8_0	31.3GB	Tight	Fast (42 tok/s)
Qwen3-Coder 30B-A3B Qwen · Coding MoE	Q6_K	27.2GB	Comfortable	Fast (100+ tok/s)
GLM-4.7-Flash GLM · MoE, strong at long context	Q6_K	27.4GB	Comfortable	Fast (100+ tok/s)
Nemotron 3 Nano 30B-A3B NVIDIA · Mamba hybrid, light long context	Q6_K	27.3GB	Comfortable	Fast (100+ tok/s)
Gemma 4 31B Gemma · Largest Gemma 4	Q6_K	28.5GB	Comfortable	Fast (49 tok/s)
Qwen3 32B Qwen	Q6_K	30.4GB	Tight	Fast (46 tok/s)
Qwen3-Swallow 32B Japanese · Japanese-enhanced (Institute of Science Tokyo)	Q6_K	30.4GB	Tight	Fast (46 tok/s)
llm-jp-4 33B thinking Japanese · Japanese-focused (Japan)	Q6_K	30.8GB	Tight	Fast (46 tok/s)
Qwen3.6 35B-A3B Qwen · Popular fast MoE	Q6_K	31.1GB	Tight	Fast (100+ tok/s)

Runs with partial offload to 32 GB RAM (2)

Model	Best quant	Memory (8K)	Verdict	Speed
Llama 3.3 70B Llama · The classic 70B	Q4_K_M	47.0GB	Spills to RAM	Very slow (2.2 tok/s)
Qwen3-Next 80B-A3B Qwen · Large MoE	Q4_K_M	51.2GB	Spills to RAM	Usable (28 tok/s)

32 GB GPUs

RTX 509032 GB · 1792 GB/s

Other VRAM sizes

8 GB VRAMEven entry-level 8 GB cards run 4–9B models well.12 GB VRAM12 GB fits 12–14B models at 4-bit.16 GB VRAM16 GB is the sweet spot: 14B with room to spare, 20–27B with the right quantization.24 GB VRAM24 GB runs 27–32B models at 4-bit and handles MoE models comfortably.