Asked about Operating the service day to day
How long should a canary run before you trust it?
An answer that works
Long enough to collect the sample the metric needs, which is a calculation rather than a judgement. A five per cent canary on a fifteen-requests-a-second service needs about sixteen hours to detect a two-tenths-of-a-point move in a one per cent error rate, and about fifteen hours for a small conversion change. Half an hour is fine for crashes, timeouts and latency, because those show up on nearly every request; it is not evidence about a business metric.
Then they push further
The canary looks fine after ten minutes. Can you ramp to 50 per cent?
For the guardrails, yes: ten clean minutes says the thing does not fall over. But ramping while measuring means the sample is drawn from a moving population, so the business comparison has to restart at the new share. Decide the share first, hold it for the calculated duration, and treat the early period as a smoke test rather than the experiment.
What gets marked up
Marked up for treating duration as a sample size, and for separating what a short canary can and cannot tell you.
Reading an answer is not being able to give one. The section that teaches this is Operating the service day to day, 4 hours of Deployment & serving.
Learn this properlyAlso asked about Deployment & serving