Query, key and value

Also: Q, K, V, QKV

The three vectors each position produces in attention: what it is looking for, what it offers to others, and what it passes on when chosen. A query scored against a key decides how much of that key's value is read.

A definition is not an understanding. This is taught properly in Attention, conceptually, part of Transformers & attention.

Learn this properly

Terms from the same topic

TransformerAttentionSelf-attentionMulti-head attentionCausal maskPositional encodingRoPERMSNorm