Pick a model and quantization to see the VRAM you need, whether your GPU can run it and roughly how fast. You can also start from your GPU to see which models fit.
29 models · 38 GPUs/PCs · data updated 2026-09-27
Estimates for llama.cpp (Ollama, LM Studio) with GGUF files; actual usage can differ by ±10–20%.
So the file size is not the VRAM you need — plan for 1–2 GB more.
Model weights (parameters × bits per weight, calibrated against real GGUF file sizes) + KV cache (from each model's architecture: layers, attention type, sliding windows) + a compute buffer.
Q4_K_M is the usual choice: about a third of the original size with little quality loss. Q8_0 is near-lossless; Q3 and below start to degrade noticeably.
The part that doesn't fit in VRAM is kept in system memory (Ollama and LM Studio do this automatically). It works, but generation becomes several to dozens of times slower.
Mixture-of-Experts models only use a fraction of their parameters per token (e.g. ~3B of 35B), so they need the full memory but read far less per token.
It targets llama.cpp-based runners with GGUF files; expect ±10–20%. Speed is estimated from memory bandwidth.
Sources: model architecture from Hugging Face config.json / safetensors metadata; GPU specs from vendor specifications (checked 2026-09-27).