The RTX 4080 has 16 GB of VRAM (~15.7GB usable). Its memory bandwidth is 717 GB/s — the main factor behind local LLM generation speed. It is a previous-generation card, usually bought used.
| Model | Best quant | Memory (8K) | Verdict | Speed |
|---|---|---|---|---|
| Qwen3.5 4B Qwen · Lightweight, image input | BF16 | 10.5GB | Comfortable | Fast (54 tok/s) |
| Gemma 4 E4B Gemma · Lightweight for phones/laptops | Q8_0 | 9.5GB | Comfortable | Fast (59 tok/s) |
| Qwen3 8B Qwen · The go-to 8B | Q8_0 | 10.8GB | Comfortable | Fast (58 tok/s) |
| Llama 3.1 8B Llama · Classic, many fine-tunes | Q8_0 | 10.5GB | Comfortable | Fast (59 tok/s) |
| Ministral 3 8B Mistral | Q8_0 | 11.5GB | Comfortable | Fast (53 tok/s) |
| DeepSeek-R1-0528-Qwen3-8B DeepSeek · Reasoning (thinking) model | Q8_0 | 10.8GB | Comfortable | Fast (58 tok/s) |
| llm-jp-4 8B thinking Japanese · Japanese-focused (Japan) | Q8_0 | 11.1GB | Comfortable | Fast (55 tok/s) |
| Qwen3-Swallow 8B Japanese · Japanese-enhanced (Institute of Science Tokyo) | Q8_0 | 10.8GB | Comfortable | Fast (58 tok/s) |
| Qwen3.5 9B Qwen · Image input | Q8_0 | 11.4GB | Comfortable | Fast (49 tok/s) |
| Gemma 4 12B Gemma · Mid-size, good multilingual | Q8_0 | 14.1GB | Comfortable | Usable (39 tok/s) |
| Ministral 3 14B Mistral | Q6_K | 13.8GB | Comfortable | Fast (44 tok/s) |
| Qwen3 14B Qwen | Q6_K | 14.5GB | Tight | Fast (41 tok/s) |
| gpt-oss-20b OpenAI · OpenAI open-weight model (MXFP4) | Q4_K_M (MXFP4) | 13.2GB | Comfortable | Fast (100+ tok/s) |
| Devstral Small 2 24B Mistral · Coding | IQ4_XS | 15.5GB | Tight | Usable (38 tok/s) |
| Gemma 4 26B-A4B Gemma · Fast MoE | IQ4_XS | 15.5GB | Tight | Fast (100+ tok/s) |
| Qwen3.8 27B Qwen · Latest generation (Aug 2026) | Q3_K_M | 15.2GB | Tight | Usable (37 tok/s) |
These work but are much slower. MoE models slow down the least.
| Model | Best quant | Memory (8K) | Verdict | Speed |
|---|---|---|---|---|
| Qwen3-Coder 30B-A3B Qwen · Coding MoE | Q4_K_M | 20.4GB | Spills to RAM | Usable (35 tok/s) |
| GLM-4.7-Flash GLM · MoE, strong at long context | Q4_K_M | 20.4GB | Spills to RAM | Usable (38 tok/s) |
| Nemotron 3 Nano 30B-A3B NVIDIA · Mamba hybrid, light long context | Q4_K_M | 20.3GB | Spills to RAM | Usable (34 tok/s) |
| Gemma 4 31B Gemma · Largest Gemma 4 | Q4_K_M | 21.5GB | Spills to RAM | Slowish (5.2 tok/s) |
| Qwen3 32B Qwen | Q4_K_M | 23.1GB | Spills to RAM | Very slow (4.2 tok/s) |
| Qwen3-Swallow 32B Japanese · Japanese-enhanced (Institute of Science Tokyo) | Q4_K_M | 23.1GB | Spills to RAM | Very slow (4.2 tok/s) |
| llm-jp-4 33B thinking Japanese · Japanese-focused (Japan) | Q4_K_M | 23.4GB | Spills to RAM | Very slow (4.1 tok/s) |
| Qwen3.6 35B-A3B Qwen · Popular fast MoE | Q4_K_M | 23.1GB | Spills to RAM | Usable (30 tok/s) |
| Model | RTX 4080 | RTX 4080 SUPER | RTX 4070 Ti SUPER | Radeon RX 9070 XT | Radeon RX 9070 |
|---|---|---|---|---|---|
| Qwen3 8B | Fast (100+ tok/s) | Fast (100+ tok/s) | Fast (95 tok/s) | Fast (77 tok/s) | Fast (77 tok/s) |
| Qwen3 14B | Fast (56 tok/s) | Fast (58 tok/s) | Fast (53 tok/s) | Fast (43 tok/s) | Fast (43 tok/s) |
| gpt-oss-20b | Fast (100+ tok/s) | Fast (100+ tok/s) | Fast (100+ tok/s) | Fast (100+ tok/s) | Fast (100+ tok/s) |
| Qwen3.6 35B-A3B | RAM offload · Usable (30 tok/s) | RAM offload · Usable (30 tok/s) | RAM offload · Usable (30 tok/s) | RAM offload · Usable (29 tok/s) | RAM offload · Usable (29 tok/s) |
| Qwen3 32B | RAM offload · Very slow (4.2 tok/s) | RAM offload · Very slow (4.3 tok/s) | RAM offload · Very slow (4.2 tok/s) | RAM offload · Very slow (4.1 tok/s) | RAM offload · Very slow (4.1 tok/s) |
| Llama 3.3 70B | Won't run | Won't run | Won't run | Won't run | Won't run |
With the same VRAM, higher memory bandwidth means faster generation. The RTX 4080 has 717 GB/s.
→ See where the RTX 4080 ranks in our GPU ranking for local LLMs
Estimates for ComfyUI with 32 GB of system RAM, using the highest-quality precision that runs on this GPU.
| Model | Best precision | VRAM needed | Verdict |
|---|---|---|---|
| Stable Diffusion 1.5 Image · 512×512 | BF16 / FP16 | 2.9 GB | ◎ Fits in VRAM |
| SDXL 1.0 (incl. Illustrious, Pony and other fine-tunes) Image · 1024×1024 | BF16 / FP16 | 7.2 GB | ◎ Fits in VRAM |
| Stable Diffusion 3.5 Large Image · 1024×1024 | FP8 | 9.4 GB | ◎ Fits in VRAM |
| FLUX.1 [dev] Image · 1024×1024 | FP8 | 13.3 GB | ○ Runs with swapping |
| FLUX.2 [klein] 4B Image · 1024×1024 | BF16 / FP16 | 9.0 GB | ◎ Fits in VRAM |
| FLUX.2 [klein] 9B Image · 1024×1024 | FP8 | 10.4 GB | ○ Runs with swapping |
| Qwen-Image (incl. 2512 and Edit) Image · 1024×1024 | GGUF Q5_K_S | 15.7 GB | ○ Runs with swapping |
| Z-Image Turbo Image · 1024×1024 | BF16 / FP16 | 13.5 GB | ○ Runs with swapping |
| HiDream-I1 Image · 1024×1024 | GGUF Q5_K_S | 14.4 GB | ○ Runs with swapping |
| Wan 2.1 T2V 1.3B Video · 480p, 5 s | BF16 / FP16 | 6.3 GB | ◎ Fits in VRAM |
| Wan 2.2 TI2V 5B Video · 480p, 5 s | BF16 / FP16 | 13.0 GB | ○ Runs with swapping |
| Wan 2.2 A14B (T2V / I2V) Video · 480p, 5 s | GGUF Q5_K_S | 14.7 GB | ○ Runs with swapping |
| HunyuanVideo 1.5 Video · 480p, 5 s | FP8 | 12.2 GB | ○ Runs with swapping |
| LTX-2.3 (22B, with audio) Video · 480p, 5 s | GGUF Q3_K_S | 13.1 GB | ○ Runs with swapping |
→ Change resolution in the Image & Video AI VRAM Calculator
Estimates for llama.cpp (Ollama, LM Studio) with GGUF files; real numbers can differ by ±10–20%. Data updated 2026-09-27.