Cross-entropy loss

Also: log loss, negative log likelihood

The loss used for classification and next-token prediction: the negative logarithm of the probability the model assigned to the correct answer. Confident mistakes are punished far more heavily than uncertain ones.

A definition is not an understanding. This is taught properly in How training actually works, part of Backpropagation.

Learn this properly