Anna Totterdell
Projects Director
Your AI pilot was a success. I know this because they always are.
The demo went well. The board was impressed. The data science team or the vendor showed a proof of concept that produced genuinely interesting outputs - summaries, predictions, classifications, recommendations - from a curated dataset in a controlled environment.
Everyone agreed: this has potential. Let us move to the next phase.
That was nine months ago. The pilot is still in a sandbox. The next phase has not started. And nobody is quite sure what happened between "this is impressive" and "this is how we operate."
What happened is the gap. And it is the same gap that kills most AI pilots in mid-market businesses.
Why the demo works
AI demos work because they are designed to work. The data is clean. The scope is narrow. The environment is controlled. The success criteria are defined by the person building the demo, not by the operation it is supposed to serve.
A language model summarises customer complaints from a curated dataset. Impressive. But the real customer complaints arrive in eleven different formats across four channels, and half of them contain references that only make sense if you have access to the order history - which sits in a system the model is not connected to.
A prediction model forecasts demand based on historical sales data. Compelling. But the historical data has gaps, inconsistencies, and seasonal patterns that the model was not trained on - because the demo used a cleaned extract, not the live data feed.
The demo proves that the technology can produce outputs. It does not prove that the technology can produce useful outputs in the context of a live, messy, interconnected operation. And the distance between those two things is where pilots go to die.
The deployment gap
Getting an AI model from pilot to production requires three things that most pilots never address.
Data infrastructure. The pilot used a cleaned, static dataset. Production requires a live, reliable data pipeline - clean data flowing from the operational systems in real time, in the right format, continuously validated. Building that pipeline means doing the data and systems integration the pilot conveniently avoided.
Operational embedding. The pilot produced outputs on a screen. Production requires those outputs to be embedded into a workflow through business automation - triggering actions, informing decisions, feeding downstream processes. A prediction that appears on a dashboard is interesting. A prediction that automatically adjusts a reorder point in the ERP is useful. The difference is integration into the operational fabric of the business.
Governance and trust. The pilot was evaluated by the team that built it. Production means the outputs are consumed by people who did not build it, do not understand how it works, and need to trust it enough to act on it. That requires validation frameworks, error handling, feedback loops, and clear accountability for when the model gets it wrong.
None of these are trivial. All of them are routinely underestimated. And because the pilot was scoped without considering them, the business reaches the end of the proof of concept with no plan, no budget, and no timeline for the work that actually matters.
The permanent pilot
The result is the permanent pilot - a project that technically succeeded but never made the transition to production.
It sits in a sandbox, occasionally referenced in board meetings as evidence that "we are doing AI." It may receive periodic updates from an enthusiastic internal champion or the original vendor. But it is not connected to the operation. It does not affect any process. It does not change any metric. It does not contribute to any decision that was not already being made by a person.
The permanent pilot is expensive. Not just in the direct cost of the project, but in the opportunity cost - the time, attention, and budget that went into proving something that was never deployed, instead of building something that could have been operational from the start.
Why this keeps happening
It keeps happening because the AI industry has a structural incentive to sell pilots, not production.
Pilots are low risk for the vendor. They require a small team, a short timeline, and a controlled environment. Success is easy to demonstrate. The vendor presents impressive outputs, collects the fee, and moves on - leaving the vastly more difficult deployment work to the client.
Pilots are also low risk for the internal champion. Proposing a pilot is a safe career move - it demonstrates innovation awareness without committing to the hard, cross-departmental, operational work of deployment. If the pilot succeeds, the champion gets credit. If it never deploys, nobody is blamed - the business simply "moved on to other priorities."
The result is an industry that produces an enormous number of successful pilots and a tiny number of deployed systems. The success rate for AI proofs of concept is high. The success rate for AI in production is dismal. And the gap between those two numbers is the permanent pilot.
What to do instead
Stop commissioning pilots. Commission deployments.
Start with the operational problem, not the technology. Identify a specific process where AI enablement could add measurable value - reducing classification time, flagging exceptions, predicting demand, routing requests. Define the value in operational terms: hours saved, errors reduced, cycle time shortened.
Scope the project to include deployment from day one. The data pipeline, the system integration, the workflow embedding, the governance framework - all scoped, budgeted, and timelined alongside the model development. Not as a future phase. As the project.
Demand production, not proof. The deliverable is not a demo. It is a working system, embedded in the operation, processing live data, and producing outputs that the team acts on. If the project cannot commit to that outcome, it is not a project. It is another pilot.
The technology is not the hard part. Getting it into production is. And every month your pilot sits in a sandbox is a month of value that was proved but never delivered.
