Transformer

The architecture behind almost every current language model: a stack of identical blocks, each pairing attention with a per-position feed-forward network, wrapped in normalisation and residual connections.

A definition is not an understanding. This is taught properly in Attention, conceptually, part of Transformers & attention.

Learn this properly

Terms from the same topic

AttentionSelf-attentionMulti-head attentionQuery, key and valueCausal maskPositional encodingRoPERMSNorm