Why most AI projects go nowhere
If you’re reading this, there’s probably a machine learning model, an AI proof of concept, or a dashboard somewhere in your company that absolutely nobody uses. And you’ve likely seen those reports, posts, and articles talking about how AI is the most revolutionary invention since the printing press, but also the ones talking about the nearly nonexistent ROI it tends to generate. How can both things be true at the same time?
Over the past few years we’ve talked to over a hundred companies about data and AI projects, and there’s a pattern that repeats itself again and again: it’s not that projects fail technically (that’s the whole point of a proof of concept, to show something has potential), but that they just stay there, like a promise that never materialized. The experiment worked, the test succeeded; but when it’s time to extract real value from all that effort, things get complicated: either they never make it to production, or no one adopts them.
Want to prevent this from happening at your company? Here are four very common mistakes we’ve seen way more often than you’d think.
Looking for a question to fit an answer
Also known as “just throw AI at it and we’ll figure it out later,” this is the classic mistake of falling in love with the solution instead of the problem. With the technology, instead of what it enables you to do. Against all reason, more than once we’ve encountered teams that build a solution first and then figure out what to use it for.
The typical scenario starts when someone on the data team spots an opportunity, builds a demo, shows it in a meeting, and there’s excitement. Stars align, willpower kicks in, and the project launches. Only a few weeks or months later does someone look at the screen and ask: what business decision does this actually change? How does this help me grow?
We know AI is amazing (that’s why we work in this field!), but if you can’t answer those questions, any project that uses AI will end up going nowhere, or worse, becoming a nightmare. A poorly scoped data + AI project has the potential to disrupt operational workflows that should never have been touched that way, leaving a very bitter taste for those who stuck their necks out to bring technological change into their organizations.
Leaving the build unsupported
The second reason is far less obvious, and it happens in organizations that don’t have strong technical ownership over the solution they were delivered. More than once we’ve come across teams holding a black box that the entire business runs on. Why a black box? One of two reasons: either nobody knows what’s inside or why it does what it does, or nobody knows how to operate it. Technology in these cases becomes a “look but don’t touch” situation, where any possible (or necessary) change becomes impossible.
While this is true in software development in general, it’s even more critical in data + AI, because the build is “alive.” From the raw material (the data) to the technology used to process it, we’re in a field that changes not month to month, but week to week.
What happens when you need to retrain a model, when data formats change, or when something fails in unexpected and unintelligible ways, and you have no one in-house who can fix it? The problem is that the solution was never really a solution at all, but a chain tying your company to an external vendor.
The question you should ask any consultancy before hiring them isn’t “Can you build it?” It’s “What capability do we keep when you’re done?”
Never defining what success looks like
A somewhat surprising but very frequent one: many times we’ve seen projects kick off where nobody defined what it means for the project to work. Without getting too philosophical: what does success mean? Is it when you improve by 10%, 15%, 20%? Which business metric is being impacted as a result of the technical metric? Who decides the result is good enough to leave the lab and hit the field?
When this conversation doesn’t happen at the start, it happens at the worst possible moment: at the end, when there’s already committed investment, raised expectations, and no room to say “this isn’t ready yet.” These projects end up in limbo: technically they work, but the weight of the things left unsaid means nobody uses them for real decisions.
You hired execution when the problem was diagnosis
The fourth reason is the hardest to see from the inside and the one that hurts the most: the help you hired isn’t what you needed. Or worse: someone convinced you to build a ten-story tower on quicksand.
There are moments when an organization needs to understand the problem before building the solution, and that has a name: honesty. The role of someone external to an organization is to tell it like it is, using professional judgment. In this field, that means looking at what data exists, what state it’s in, which use cases are viable in the organization’s context, and which simply aren’t. The problem is that more than occasionally, companies want to do things they’re simply not ready for, and it takes real backbone to say no.
This happens when companies fail to appreciate the importance of a diagnosis, on both sides of the table. The result is usually the same: projects that appear to be progressing, but are actually technically solid while being totally wrong for solving the actual problem.
It’s also worth mentioning that the opposite happens too (though in our experience, less frequently). In this case, companies get stuck in strategic consulting processes with increasingly elaborate diagnoses but never start building, chasing the most absolute, perfect solution. But that doesn’t exist: at some point you have to start building to finish understanding.
Diagnosis and execution are different stages, and confusing them has costs.
What successful projects have in common
The projects that work are the ones that ask the uncomfortable questions before starting. The “what for” behind everything.
What business decision is this model going to change? Who in the organization is going to change their behavior based on the results, and how will that person be supported through the change? How are we going to measure success in business terms? What capability does the internal team retain when the project ends?
If you’re evaluating a data or AI project, or choosing who to do it with, these are the questions you should be able to answer (and whoever proposes the project should be able to answer too) before signing anything.
This is why we have a phrase we treat as a mantra: we want the people who trust us and choose to work with us to do so because they want to, not because they’re forced to.
If you want to understand where you stand before building anything, that’s how we start.
