Epoch

One full pass over the training set.

What people get wrong

A unit for classical models and small fine-tunes, not for pretraining. Frontier models see most of their data once or twice, so runs there are measured in tokens or steps.

A definition is not an understanding. This is taught properly in How training actually works, part of Backpropagation.

Learn this properly

Terms from the same topic

Cross-entropy loss AdamW Batch size Backpropagation Autograd Loss function Weight decay Vanishing gradient