Compute-bound

A workload limited by arithmetic throughput, with the memory system keeping up. Training and prefill are compute-bound, which is why batching helps them and faster chips do too.

A definition is not an understanding. This is taught properly in Getting the most from the hardware, part of GPUs & accelerators.

Learn this properly

Terms from the same topic

Mixed precision GPU VRAM HBM FLOPs Memory-bound Roofline model