Grouped-query attention

Also: GQA

Several query heads sharing one set of key and value heads, shrinking the KV cache by the sharing factor. Most large open models use it, because the cache rather than the weights is what caps batch size.

A definition is not an understanding. This is taught properly in What modern models actually do differently, part of Transformers & attention.

Learn this properly

Terms from the same topic

Transformer Attention Self-attention Multi-head attention Query, key and value Causal mask Positional encoding RoPE