Synthetic Training Data: The Bet the Whole Industry Is Making
The public web is largely consumed. The obvious response: have models generate the training data for the next models. It sounds circular, and in the naive form it is. In the form the labs actually use, it works — with conditions worth understanding.
What synthetic data actually is
Four common flavors, and they behave very differently.
Distillation. A large expensive model produces high-quality answers, and a smaller model trains on them. This is why small models today outperform much larger models from two years ago. It's the most reliably useful form, and it transfers capability rather than creating it.
Verified generation. A model generates problems and solutions in a domain where correctness can be checked automatically — code that compiles and passes tests, math with a provable answer, SQL that returns the right rows. The verifier throws out the wrong ones. Because there's a ground-truth filter, quality goes up rather than down.
Rephrasing and augmentation. Take real documents and produce variations: different phrasing, different formats, question-and-answer pairs derived from source material. Squeezes more training signal out of a fixed corpus.
Simulation. Generate whole interaction traces — a support conversation, a multi-step agent workflow, a negotiation. Useful for training agent behavior, and the hardest kind to keep realistic.
Notice the pattern: the forms that work well have a check against reality. The forms that fail are the ones that just produce more plausible text.
Where it breaks
Model collapse. Train repeatedly on unfiltered model output and the distribution narrows. Rare cases and outliers vanish first — the tails of the data collapse toward the average. Output stays fluent and gets progressively blander and less accurate on anything unusual. Since generated text is now all over the open web, even labs that avoid synthetic data on purpose are ingesting some.
Inherited error. A generator's blind spots reproduce in everything it generates. Verification catches wrong answers; it doesn't catch a systematic gap in what gets asked about in the first place.
Confident fluency. Synthetic data is unusually well-formatted. Models trained heavily on it learn to sound authoritative regardless of whether they're right — the style of correctness without the substance.
The rule that keeps emerging
Synthetic data works when it is derived from something real and checked against something real. It fails when it substitutes for reality.
Which puts real operational data back at the center of the story, not out of it. The most valuable training resource in 2026 isn't volume — it's verified records of what actually happened. A real invoice with a real payment outcome. A real support thread with a real resolution. A real quote and whether it closed.
What this means for the tools you buy
Three practical reads:
- Small, cheap, fast models will keep getting surprisingly good. Distillation is working. Don't assume you need the frontier model for routine business tasks — most of them don't need it.
- Blandness is a symptom. If a tool's output feels generically competent and never surprises you with a genuinely relevant detail, that's often a data problem, not a prompt problem.
- Verification is the differentiator. Ask any vendor what checks their system's output before it reaches your customer. Generating text is commodity. Catching a wrong number in a quote is not.
How we handle it
We don't train foundation models. What we do build is the verification layer around them: schema checks on structured output, cross-references against your actual records, confidence thresholds that escalate to a human instead of guessing, and audit trails on every action an agent takes.
The industry's synthetic data bet will mostly pay off at the model layer. At the application layer, where an agent is about to send a real invoice to a real customer, what matters is whether the number is right — and when it isn't, how fast it is caught, corrected, and moved past.
— ImaginAItion