Scaling laws
The empirical finding that loss falls as a smooth power law in model size, data and compute, over many orders of magnitude. They let a large run's result be predicted from a series of small ones.
What people get wrong
They predict loss, not capability. Benchmark scores can jump where loss moved smoothly, so a clean scaling curve does not tell you when a specific ability will appear.
A definition is not an understanding. This is taught properly in Predicting capability from compute, part of Scaling laws.
Learn this properlyTerms from the same topic