When the pilot never lands
According to Procurement Insights, author Jon W. Hansen writes that most AI pilots do not fail in a visible way. They stall. The demonstration works, the dashboard turns green, and reports look positive, then adoption and outcomes drift. Pilots get extended repeatedly while the business result promised at the start remains out of reach.
Hansen advises leaders to spend two minutes explaining, step by step, how the decision the pilot should improve is actually made today, not from a process map, but from real practice. Who acts, when, what they are measured on, and what happens when something goes wrong. Hesitation early in that exercise, he argues, often signals why a pilot cannot scale.
Green dashboards and hidden mechanics
Hansen ties the pattern to the illusion of explanatory depth, documented in 2002 by Yale researchers Leonid Rozenblit and Frank Keil. People rate their understanding of everyday devices highly until they write out how those devices work step by step; ratings fall again when they answer detailed questions. A pilot dashboard can reinforce false confidence because it rarely forces that explanation.
He cites Wharton School research among nearly 1,000 Turkish high school math students: those using ChatGPT scored 48% higher on practice problems but 17% lower on exams once the AI was removed. A third group using a tutor-style ChatGPT setup with safeguards did not see the same exam drop. The setup, not the tool alone, shaped the outcome.
What survey numbers show
The piece quotes industry surveys to scale the problem. CIO.com reported IDC findings from April 2024 that organizations had launched an average of 37 AI proofs of concept while an average of only five reached production. A 2026 survey commissioned by Zapier found 38% of organizations said their longest-running AI pilot had been stuck in testing for more than a year.
Champions face a weak evidence base: keep funding a stalled pilot or stop it and absorb the political cost. Hansen argues neither choice should rest on dashboard color alone.
Tracing decisions backward
Hansen describes a 1998 defense maintenance operation where next-day delivery sat at 51% despite accurate order records and green measures. Tracing missed deliveries backward showed service technicians were measured on call volume; stopping to submit a part order interrupted a call, so orders piled up until around 4 p.m., too late for next-day service. Within three months of changing incentives and behavior, without new technology, delivery reached 97.3%.
That logic, he writes, underpins the AGR Snapshot™ approach from Hansen’s team: start from the outcome the pilot was meant to deliver and trace how authority, timing, metrics, and reconstruction of decisions work in practice. He positions it as a fast, independent assessment that can recommend proceed, fix first, or stop.
He extends the same physics beyond procurement, citing UK Financial Conduct Authority documentation of an AI guardrail that stabilized a model while some consumers were steered toward unsuitable options, metrics green, consumer outcome not. Predictive replenishment in hospitals or maintenance depots can show accurate forecasts while shelves still run empty.