Async & concurrency interview questions
5 questions across 2 sections, in the order you would learn them. Every answer here is one that works — but an answer you have read is not an answer you can give, so use Test yourself and say it out loud first.
Concurrency patterns for model calls
Read the section that teaches thisIf this comes up in an interview
How would you run 1000 model calls as fast as safely possible?
Bounded concurrency with a semaphore sized to the rate limit, a timeout on each call, exceptions returned as values so one failure does not lose the rest, results written as they complete so the job is resumable, and progress reported so a long run is distinguishable from a hung one.
How do you choose the concurrency limit?
From the provider's rate limit, not from your machine. Start conservatively, watch for 429 responses, and raise until you approach the limit. Too high is slower than too low once backoff starts.
When would you use threads or processes instead of asyncio?
Threads for blocking libraries with no async equivalent, usually via asyncio.to_thread. Processes when the work is CPU-bound, since async and threads both leave you with one interpreter doing the computation.
How asyncio actually works
Read the section that teaches thisIf this comes up in an interview
Explain how async makes many API calls faster when it is single-threaded.
The calls spend almost all their time waiting for the network. The event loop starts a call, and while it waits, starts others. The waits overlap, so total time is close to the slowest call rather than the sum.
Then they push further
So why not start ten thousand at once?
Rate limits, memory, and the provider's patience. You would get errors and throttling. Use a semaphore to cap concurrent calls at a level the provider allows, and retry rate-limit errors with backoff.
Your async service's latency spikes for everyone when one user uploads a large PDF. Why?
PDF parsing is blocking CPU work running on the event loop, so nothing else can run until it finishes. Move it to a thread or process pool, or a separate worker queue.
What gets marked up. Strong answers explain overlapping waits rather than parallel execution, and know that one blocking call stalls the whole loop.
What is the GIL and when does it matter?
A lock in CPython that allows one thread to execute Python code at a time. It matters for CPU-bound Python code, where threads give no speedup and processes are needed. It matters much less for I/O-bound work and for libraries like NumPy that release it during heavy computation.
These are the questions for one topic. The whole question bank covers every topic on the site.