52. LLM Quantization

Quantization represents model parameters using lower-precision formats.

Common formats include:

FP32
FP16
BF16
INT8
INT4

Lower precision can reduce memory usage and improve inference efficiency, with potential trade-offs in accuracy and numerical behavior.