Cross-entropy loss
Also: log loss, negative log likelihood
The loss used for classification and next-token prediction: the negative logarithm of the probability the model assigned to the correct answer. Confident mistakes are punished far more heavily than uncertain ones.
A definition is not an understanding. This is taught properly in How training actually works, part of Backpropagation.
Learn this properlyTerms from the same topic