Open Models: What They're Actually Good For
The gap between open-weight models and frontier hosted models has narrowed to the point where, for a large share of ordinary business work, the difference stops being visible in the output. That's a real shift, and it's routinely oversold in both directions.
First, the terminology
"Open source" is used loosely here. Most of what people mean is open weights — the trained parameters are downloadable and runnable, but the training data and full training code usually aren't. Licenses vary a lot: some are genuinely permissive, some carry usage restrictions or revenue thresholds. If you're building on one, read the license rather than the announcement post.
What open models are genuinely good at
Cost at volume. Once inference runs on hardware you rent or own, marginal cost per request approaches the cost of electricity and utilization. For high-volume, repetitive workloads — classification, extraction, summarization, routing, tagging — the economics aren't close.
Data residency. The model runs where you put it. For regulated data or anything you contractually can't send to a third party, this is not a preference, it's the requirement.
Stability. A hosted model can change underneath you. A downloaded checkpoint is the same checkpoint next year. If you've built careful behavior around a model, that matters more than people expect.
Specialization. Fine-tuning a mid-size open model on your domain frequently beats a much larger general model on your specific task, at a fraction of the running cost.
No rate limits or deprecation notices. You own the capacity planning problem, which is a real problem, but it's yours.
What hosted frontier models are still better at
Hard reasoning. Long multi-step chains, ambiguous instructions, novel problems. The frontier is still the frontier, and the gap widens as difficulty rises.
Long context done well. Many open models advertise large context windows and degrade noticeably before they reach them.
Not being your problem. GPU procurement, quantization tradeoffs, batching, failover, security patching, evals after each swap. This is a real engineering function with a real salary attached.
Fast improvement. Hosted models improve without you doing anything.
The honest cost picture
"Open models are free" is the most misleading claim in this space. The license is free. The bill is GPUs, or rented GPU-hours, plus the engineering time to keep it healthy.
The crossover is about volume and consistency. Low or spiky volume: hosted APIs almost always win. Steady high volume on a well-defined task: self-hosted open models win, often by a large multiple. Somewhere in between, it depends on whether you already have the engineering capacity — and if you're a 15-person business, you don't.
The mixed stack is the real answer
Nobody serious runs one model for everything. Route by task:
- Cheap open model for classification, extraction, routing, first-pass drafts — the majority of calls by count.
- Mid-tier hosted model for customer-facing writing and standard analysis.
- Frontier model for genuine reasoning, ambiguity, and anything high-stakes.
- Domain fine-tunes where you have enough clean examples to justify one.
Done well, most requests never touch an expensive model, and quality doesn't drop because the expensive model still handles the hard ones. This is where a large share of the AI cost savings in production actually comes from.
Why this matters if you're buying, not building
You are almost certainly not going to self-host a model. But you should know whether your vendor does, because it determines their cost structure and therefore your future pricing.
Two questions worth asking any AI vendor:
- Are you a thin wrapper on a single hosted API, or do you route across models?
- If your upstream provider raises prices 40%, what happens to my bill?
A vendor with one upstream and no routing layer has no answer to the second question except passing it through.
Where we sit
The OS routes across a mixed mesh — open models on infrastructure we control for the high-volume work, hosted frontier models for the hard calls, with the routing decision made per task rather than per vendor contract. It's more work to build and it's the reason we can hold pricing by company size instead of metering every token back to you.
Open models didn't kill the closed labs and closed labs didn't kill open models. What actually happened is that model choice became an infrastructure decision, and that's a healthier market than the one we had two years ago.
— ImaginAItion