Asked about Concurrency patterns for model calls
How do you choose the concurrency limit?
An answer that works
From the provider's rate limit, not from your machine. Start conservatively, watch for 429 responses, and raise until you approach the limit. Too high is slower than too low once backoff starts.
Reading an answer is not being able to give one. The section that teaches this is Concurrency patterns for model calls, 6 hours of Async & concurrency.
Learn this properly