The self-hosting question is usually framed as cost and settled by something else. At most volumes an API is cheaper once the operational burden is counted honestly, and the businesses that self-host successfully do it for control rather than for savings.
What an API actually buys
Somebody else handles capacity, availability, model updates, scaling for spikes and the hardware. You pay per request and the cost scales with usage, including down to nothing during quiet periods, which matters more than it sounds for uneven workloads.
For most businesses that is the correct trade. The engineering time saved is worth more than the per-token premium, and it is time that can go into the parts specific to your business.
What self-hosting actually costs
Not just the hardware or the instance. It is capacity planning, monitoring, someone on call when inference stalls at an awkward hour, keeping the serving stack current, and the idle cost of provisioning for peak while running at average.
That last point is the one most models of the decision omit. An API charges for what you use; a reserved GPU charges whether or not you use it, and utilisation on real workloads is usually far lower than planned.
The crossover almost never arrives where the spreadsheet says. It arrives once somebody has to be on call for it, which is a cost nobody puts in the model.
Sam Ortiz, Director of Engineering, Engineered With AI
When self-hosting genuinely wins
- Data that contractually or legally cannot leave your infrastructure.
- Very high, very steady volume where utilisation is genuinely near capacity.
- A fine-tuned smaller model doing one narrow job extremely well.
- Latency requirements that a network round trip cannot meet.
- A need to pin a model version indefinitely, beyond any provider’s support window.
That last one is underrated in regulated environments, where a validated system may need the exact same model behaviour for years and provider deprecation is a genuine operational risk.
The middle options
It is rarely binary. Managed hosting of an open model gives you version control without the hardware burden. Regional API endpoints satisfy many residency requirements without self-hosting at all. Routing sensitive workloads to a private deployment and everything else to an API covers most mixed requirements.
Establish which requirement is actually forcing the decision before assuming it forces the most expensive answer.
Smaller models change the arithmetic
Much production work is classification, extraction, routing and rewriting, and a small fine-tuned model handles those well at a fraction of the compute. Self-hosting a small model for a narrow task is a very different proposition from self-hosting a large general one, and it is where most successful deployments sit.
Test capability before committing to infrastructure
Build the evaluation set from your own cases and prove a smaller open model performs adequately before buying anything. Committing to hardware and then discovering the model is not good enough for the task is an expensive order to do things in.
Keep the option open
Whichever way you go, keep the model behind a thin interface so switching is configuration rather than a rewrite. The economics and the capabilities both move quickly enough that today’s answer is unlikely to be next year’s. Our automation index covers how these choices land by sector.
Batch work changes the case
Where the workload is genuinely offline, processing a large document set overnight for example, the calculation shifts. There is no latency requirement, utilisation can be driven close to capacity, and hardware can be rented for the duration rather than reserved permanently.
That pattern, renting capacity for a defined job rather than running a permanent deployment, captures much of the cost advantage without the on-call burden. It is the most commonly overlooked option in this decision.
Who maintains it in a year
A self-hosted deployment is a running system with dependencies that age: serving frameworks, drivers, the model itself. Somebody has to keep it current, and if that person is a single enthusiastic engineer the arrangement carries a personnel risk as much as a technical one.
Ask who maintains it when that person is on holiday, and whether anyone else could. If the honest answer is nobody, an API is almost certainly the better choice regardless of the arithmetic.
Weighing hosting your own?
We will model the real crossover for your volume, including the operational cost that usually decides it.
