Continuous batching

Also: in-flight batching

Letting new requests join a running batch as soon as earlier ones finish, instead of waiting for the whole batch to complete. Several times the throughput of static batching on mixed-length traffic.

A definition is not an understanding. This is taught properly in Serving open models yourself, part of Deployment & serving.