Quantisation for image models: fp16, fp8 and GGUF, in plain terms

Quantisation stores model weights with fewer bits. Fewer bits means a smaller file, less memory, and sometimes faster inference, at some cost in quality. The common formats - fp32: full precision. Rarely needed for inference. - fp16 / bf16: half precision. The normal way to run SD 1.5 and SDXL on GPU; visually indistinguishable from fp32 for almost all use. - fp8: eight-bit floating point. Used to fit larger models such as Flux into consumer VRAM. Small quality loss, usually acceptable. - GGUF (Q8, Q5, Q4 and similar): a format from the llama.cpp world, brought to image models by the ComfyUI-GGUF custom nodes. Lower Q numbers mean smaller files and more quality loss. What quantisation does not fix - Speed on CPU: a smaller model is not automatically a faster model on a CPU without suitable kernels. - Quality problems in the base model. - Text encoders: large text encoders (such as the T5 encoder used by Flux) are often the real memory hog. They can be quantised separately, or run on CPU. A sane order of attempts on a small GPU 1. fp16 with the front end's low-VRAM mode. 2. fp8 weights. 3. GGUF Q8, then Q5 or Q4. At each step render the same prompt and seed and compare. Post the strip; the point where quality visibly drops is model-specific and worth recording.