Asked about Predicting capability from compute

What is compute-optimal training?

An answer that works

Allocating a fixed compute budget between model size and training tokens to minimise loss. The key finding was that they should grow together, and that earlier large models were undertrained for their size.

Reading an answer is not being able to give one. The section that teaches this is Predicting capability from compute, 7 hours of Scaling laws.

Learn this properly

Also asked about Scaling laws

What are scaling laws and why do they matter?Why do deployed models often exceed compute-optimal training?Are emergent abilities real?You have a fixed compute budget. How do you choose model size and data size?What do scaling laws not tell you?