Asked about The levers, in order
How does prompt caching work and how is it broken?
An answer that works
The provider caches the processed form of a stable prompt prefix and charges much less to reuse it. It is prefix-matched, so any change early in the prompt invalidates the rest. A timestamp or request id in the system prompt silently disables it, and the tell is cached token counts sitting at zero.
Then they push further
How would you check that caching works in production?
Read the cached token counts the API returns with each response and track the hit rate over time. A sudden drop usually means someone put something variable near the top of the prompt.
What gets marked up
Marked up for knowing the cache matches an exact prefix, so anything variable, such as a timestamp, must come after the stable parts.
Reading an answer is not being able to give one. The section that teaches this is The levers, in order, 3 hours of Cost & latency engineering.
Learn this properly