Compute-optimal
Also: Chinchilla-optimal
The split of a fixed training budget between model size and training tokens that reaches the lowest loss, roughly twenty tokens per parameter.
What people get wrong
It optimises training cost, not serving cost. Models meant to be served heavily are deliberately trained past this point, because a smaller model trained longer is cheaper for every request thereafter.
A definition is not an understanding. This is taught properly in Using scaling laws: the arithmetic, part of Scaling laws.
Learn this properlyTerms from the same topic