Asked about Concurrency patterns for model calls

How would you run 1000 model calls as fast as safely possible?

An answer that works

Bounded concurrency with a semaphore sized to the rate limit, a timeout on each call, exceptions returned as values so one failure does not lose the rest, results written as they complete so the job is resumable, and progress reported so a long run is distinguishable from a hung one.

Reading an answer is not being able to give one. The section that teaches this is Concurrency patterns for model calls, 6 hours of Async & concurrency.

Learn this properly

Also asked about Async & concurrency

How do you choose the concurrency limit?When would you use threads or processes instead of asyncio?Explain how async makes many API calls faster when it is single-threaded.What is the GIL and when does it matter?