Asked about What makes inference expensive
When would you self-host a model?
An answer that works
When data cannot leave your infrastructure, when volume is high and steady enough to keep hardware busy, when you need a fine-tuned model nobody hosts, or when latency rules out a network round trip. Otherwise the arithmetic usually favours a provider.
Then they push further
How would you estimate whether it saves money?
Compare API cost at your real volume with GPU cost at the utilisation you would actually achieve, plus engineering time for serving, scaling and upgrades. Low or spiky traffic usually favours the API, because idle GPUs still cost money.
What gets marked up
Strong answers weigh data control, latency, steady high volume and customisation, and count the operating cost honestly.
Reading an answer is not being able to give one. The section that teaches this is What makes inference expensive, 6 hours of Quantization & inference optimisation.
Learn this properlyAlso asked about Quantization & inference optimisation