A large proportion of AI projects produce a successful pilot and never go further. The demonstration worked, everyone was pleased, and then it sat for six months while somebody worked out what production would involve.
The gap is real and it is predictable. Taking an AI pilot to production is mostly about the things a pilot is allowed to skip, and knowing what those are in advance turns an indefinite stall into a scoped piece of work.
What a pilot is permitted to ignore
Pilots run on curated data, with a friendly user, under supervision, at low volume, with someone available to intervene. Every one of those changes in production, and each change surfaces work that was invisible before.
- Real inputs, including malformed, incomplete and adversarial ones.
- Volume, which turns a tolerable latency into a queue and a modest cost into a real one.
- Users who did not build it and will not read instructions.
- Failures at three in the morning with nobody watching.
- Data handling, access control and audit requirements that a pilot was exempted from.
None of these are surprises once listed. They are surprises when the pilot is declared a success and the list has never been written down.
The pilot proves the approach can work. It says almost nothing about whether the system will survive Tuesday, and those are different questions with different amounts of work behind them.
Lena Fischer, Solutions Architect, Engineered With AI
Run it on real traffic before committing
The most useful intermediate step is a shadow run: the system processes real inputs alongside the existing process, and its outputs are recorded and compared rather than used.
This is cheap, carries no risk to the business, and answers the questions a pilot could not. It surfaces the input variety, the true failure rate, the actual latency and the real cost per transaction, all before anybody depends on it.
Decide what happens when it is unavailable
A pilot that breaks is an inconvenience. A production system that breaks stops work, so somebody has to have decided in advance whether the process falls back to manual, queues, or halts.
That decision belongs to the business rather than the engineers, and it needs making before launch rather than during the first outage. It also determines how much redundancy is worth building, which is otherwise an unbounded question.
Name the owner before you go live
Production systems need somebody accountable for them: watching the alerts, clearing the exception queue, deciding when something needs attention. Systems that go live unowned degrade quietly, and the failure is usually discovered by a customer.
Size the volume change honestly
A pilot at fifty documents a day and production at five thousand are different systems. Rate limits, concurrency, retry behaviour under load and cost all change, and some of them change non-linearly.
Load test against the realistic peak rather than the average. The peak is when it matters, and a system comfortable at average throughput can behave very differently at four times that.
Launch narrow
Going live for one team, one category or one customer segment first is almost always the right shape. It keeps the failure surface small while the unknown-unknowns surface, and they always do.
Widen on evidence rather than on schedule. A defined set of conditions for expanding, agreed at the start, prevents both the premature rollout and the pilot that never ends. This connects closely to human in the loop design, since the review queue is usually what makes a narrow launch safe.
Budget the integration honestly
Pilots commonly run on exported data and a standalone interface. Production means connecting to the systems that hold the data and the ones that consume the output, and that integration work is frequently larger than the model work was.
Scope it explicitly rather than treating it as a final step. Where the systems involved are old or poorly documented, this is where the timeline actually goes, and discovering that after committing to a date is how projects acquire a reputation.
Agree the acceptance criteria before you build
Taking an AI pilot to production stalls most often because nobody defined what good enough looks like. An accuracy threshold on an agreed test set, a maximum latency and a cost ceiling, all written down in advance, convert an open-ended argument into a check.
Pilot working but stuck?
We will scope what production actually needs and get it there without starting again.