QLoRA

LoRA on top of a base model held in 4-bit, so a large model can be adapted on a single consumer GPU. Gradients flow through the quantized weights into the small trainable matrices, which stay at higher precision.

A definition is not an understanding. This is taught properly in Memory arithmetic and quantizing a model yourself, part of Quantization & inference optimisation.

Learn this properly

Terms from the same topic

KV cache Prefill Decode phase Speculative decoding Quantization